bug#9145: BUG in dd when pushing output over ssh
On 07/21/2011 07:36 PM, Andrew Lisciandrello wrote: Hello, I came across this bug, I don't know what it means but I figured you may want to see it. Text States: BUG: Bad page state in process dd pfm:98a00 page:ea0002163000 flags:4000 count:0 mapcount:0 mapping:02000 Unfortunately, that means you encountered a kernel fault, perhaps due to an I/O error while trying to read or write /dev/sda. That message is printed by your kernel, not by dd, so there's nothing we can do on this list to help you with it. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
bug#9145: BUG in dd when pushing output over ssh
tag 9145 unreproducible thanks On 07/22/2011 07:09 AM, andrewlisciandre...@gmail.com wrote: Unfortunately, that means you encountered a kernel fault, perhaps due to an I/O error while trying to read or write /dev/sda. That message is printed by your kernel, not by dd, so there's nothing we can do on this list to help you with it. I'm not really worried about it, especially as an md5 of both drives shows as equal after the dd was over. Feel free to mark this resolved, I just wanted to be sure you had the error message in case it was useful to you in any way. Good to hear; I'm re-adding the list and marking the bug closed. Also, thanks for working on kickass utils! Our pleasure! -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
bug#9129: Built-in printf Sits Awkwardly with UDP.
retitle 9129 printf: RFE: reject field width larger than INT_MAX tags 9129 notabug thanks Eric Blake wrote: [adding coreutils] On 07/20/2011 07:34 AM, Ralph Corderoy wrote: BTW, the code for the built-in printf has a bug. For negative field-widths it negates a negative integer without checking it will fit. E.g. on this 64-bit machine $ printf '%-9223372036854775808s.\n' foo foo. $ Coreutils' printf shares this misfortune. Sadly, it might even be a bug in the underlying glibc printf(), although I haven't tried to write a test program to check that, yet. This is not about a negative field width. You can induce misbehavior without the -. The - is just the left-alignment option. Compare these: $ printf '_%2s_\n' o _ o_ $ printf '_%-2s_\n' o _o _ The trouble is that whenever you specify a field width larger than INT_MAX, you enter into unspecified (or was it undefined?) territory. Applications like printf or bash's built-in printf should be able to detect and diagnose such questionable inputs before passing the offending directive to an underlying *printf function. I've marked this not a bug but have left the ticket open, since I see it as a reasonable request for improvement.
bug#9129: Built-in printf Sits Awkwardly with UDP.
OK, well for %b and %q bash's built-in printf calls it's own printstr() and that does do things like `fw = -fw' without checking if fw was already the largest negative. On a related note, I can't interrupt this, e.g. Ctrl-C. printf '%-92233720368547758q.\n' foo Cheers, Ralph. P.S. Please keep me CC'd.
bug#9129: Built-in printf Sits Awkwardly with UDP.
Hi Jim, On 07/20/2011 07:34 AM, Ralph Corderoy wrote: BTW, the code for the built-in printf has a bug. For negative field-widths it negates a negative integer without checking it will fit. E.g. on this 64-bit machine $ printf '%-9223372036854775808s.\n' foo foo. $ Coreutils' printf shares this misfortune. Sadly, it might even be a bug in the underlying glibc printf(), although I haven't tried to write a test program to check that, yet. This is not about a negative field width. Just to ensure there's no confusion, as far as bash's built-in printf is concerned one aspect of it is about a negative field width since $ printf '.%*q.\n' -10 foo .foo . is implemented by converting the -10 into a number, spotting it's negative, setting the left-justified flag, and then negating the number. Coreutils' printf doesn't have that particular issue since it passes the negative number to the C library as a negative and has it deal with the justification change. For %b, which libc doesn't support, coreutils says /* FIXME: Field width and precision are not supported for %b, even though POSIX requires it. */ so there's no code to go wrong. :-) Cheers, Ralph.
bug#9141: [PATCH 1/3] extensions: Enable extensions on MacOS X 10.5 and later.
Recent versions of MacOS seem to have a _DARWIN_C_SOURCE flag that has roughly the same role that _GNU_SOURCE has for GNU systems. I pushed the following patch into gnulib, under the theory that it is more in the line with the usual Autoconf / gnulib philosophy, and it may fix some porting issues. Herb, can you please try configuring coreutils with this change? If you don't have all the autotools installed, you can simply append #define _DARWIN_C_SOURCE 1 to lib/config.h after running configure. Thanks. * m4/extensions.m4 (AC_USE_SYSTEM_EXTENSIONS): Define _DARWIN_C_SOURCE. --- ChangeLog|5 + m4/extensions.m4 |7 ++- 2 files changed, 11 insertions(+), 1 deletions(-) diff --git a/ChangeLog b/ChangeLog index b22ea11..46d8d04 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2011-07-22 Paul Eggert egg...@cs.ucla.edu + + extensions: Enable extensions on MacOS X 10.5 and later. + * m4/extensions.m4 (AC_USE_SYSTEM_EXTENSIONS): Define _DARWIN_C_SOURCE. + 2011-07-22 Kamil Dudka kdu...@redhat.com file-has-acl: use acl_extended_file_nofollow if available diff --git a/m4/extensions.m4 b/m4/extensions.m4 index 1330503..22156e0 100644 --- a/m4/extensions.m4 +++ b/m4/extensions.m4 @@ -1,4 +1,4 @@ -# serial 9 -*- Autoconf -*- +# serial 10 -*- Autoconf -*- # Enable extensions on systems that normally disable them. # Copyright (C) 2003, 2006-2011 Free Software Foundation, Inc. @@ -67,6 +67,10 @@ AC_BEFORE([$0], [AC_RUN_IFELSE])dnl #ifndef _ALL_SOURCE # undef _ALL_SOURCE #endif +/* Enable general extensions on MacOS X. */ +#ifndef _DARWIN_C_SOURCE +# undef _DARWIN_C_SOURCE +#endif /* Enable GNU extensions on systems that have them. */ #ifndef _GNU_SOURCE # undef _GNU_SOURCE @@ -95,6 +99,7 @@ AC_BEFORE([$0], [AC_RUN_IFELSE])dnl test $ac_cv_safe_to_define___extensions__ = yes AC_DEFINE([__EXTENSIONS__]) AC_DEFINE([_ALL_SOURCE]) + AC_DEFINE([_DARWIN_C_SOURCE]) AC_DEFINE([_GNU_SOURCE]) AC_DEFINE([_POSIX_PTHREAD_SEMANTICS]) AC_DEFINE([_TANDEM_SOURCE]) -- 1.7.4.4
bug#9140: [PATCH 2/3] large-inode: New module
Here's another module I'm adding to coreutils. I plan to change coreutils to use it. Another possibility, I suppose, would be to fold it into AC_SYS_LARGEFILE. Herb, can you please try out the effect of this module by appending #define _DARWIN_USE_64_BIT_INODE 1 to lib/config.h after running 'configure'? Thanks. * MODULES.html.sh: Add it. * modules/large-inode, m4/large-inode.m4: New files. --- ChangeLog |4 MODULES.html.sh |1 + m4/large-inode.m4 | 24 modules/large-inode | 22 ++ 4 files changed, 51 insertions(+), 0 deletions(-) create mode 100644 m4/large-inode.m4 create mode 100644 modules/large-inode diff --git a/ChangeLog b/ChangeLog index 46d8d04..b95e9d6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,9 @@ 2011-07-22 Paul Eggert egg...@cs.ucla.edu + large-inode: New module + * MODULES.html.sh: Add it. + * modules/large-inode, m4/large-inode.m4: New files. + extensions: Enable extensions on MacOS X 10.5 and later. * m4/extensions.m4 (AC_USE_SYSTEM_EXTENSIONS): Define _DARWIN_C_SOURCE. diff --git a/MODULES.html.sh b/MODULES.html.sh index bcd8aac..80befa9 100755 --- a/MODULES.html.sh +++ b/MODULES.html.sh @@ -2656,6 +2656,7 @@ func_all_modules () func_module file-set func_module hash-triple func_module i-ring + func_module large-inode func_module same-inode func_end_table diff --git a/m4/large-inode.m4 b/m4/large-inode.m4 new file mode 100644 index 000..b41a391 --- /dev/null +++ b/m4/large-inode.m4 @@ -0,0 +1,24 @@ +# Enable large inode numbers on systems normally without them. -*- Autoconf -*- + +# Copyright (C) 2011 Free Software Foundation, Inc. +# This file is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# written by Paul Eggert + +AC_DEFUN([gl_SYS_LARGE_INODE], +[ + dnl Many systems enable large inodes if you enable large offsets. + AC_REQUIRE([AC_SYS_LARGEFILE]) + + dnl Some Mac OS X variants won't access large inode numbers by default. + dnl Defining _DARWIN_USE_64_BIT_INODE fixes this. See + dnl http://developer.apple.com/library/mac/releasenotes/Darwin/SymbolVariantsRelNotes. + dnl + dnl The simplest thing is to define this symbol everywhere. + dnl That helps on the affected systems, and doesn't hurt anywhere. + AC_DEFINE([_DARWIN_USE_64_BIT_INODE], [1], +[Define if you want to use large inode numbers + when running on Mac OS X 10.5 or later.]) +]) diff --git a/modules/large-inode b/modules/large-inode new file mode 100644 index 000..de067ac --- /dev/null +++ b/modules/large-inode @@ -0,0 +1,22 @@ +Description: +Use large inode numbers. + +Files: +m4/large-inode.m4 + +Depends-on: + +configure.ac-early: +AC_REQUIRE([gl_SYS_LARGE_INODE]) + +configure.ac: + +Makefile.am: + +Include: + +License: +LGPLv2+ + +Maintainer: +all -- 1.7.4.4
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Thanks for the further information about the problem with MacOS X 10.7.
Can you please try this patch? If you don't have autotools installed,
you can edit 'configure' by hand and install a similar patch there.
Thanks.
I have pushed this into gnulib as it seems pretty simple. The basic
idea is to avoid statvfs on MacOS X because its block counts can't
go above 2**31.
From 3d2ecb41c4a3e428e8a455c4f5fef49f50144f0f Mon Sep 17 00:00:00 2001
From: Paul Eggert egg...@cs.ucla.edu
Date: Fri, 22 Jul 2011 10:07:18 -0700
Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems
* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
implementations that use only 32 bits to count blocks.
On typical hosts with 1024-byte blocks, this fails with file
systems as small as 4 TiB. Problem reported by Herb Wartens
http://debbugs.gnu.org/9140 and this should also fix a similar
problem reported by Tim Spriggs http://debbugs.gnu.org/7355.
---
ChangeLog |8
m4/fsusage.m4 | 18 +-
2 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index b95e9d6..b652f25 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,13 @@
2011-07-22 Paul Eggert egg...@cs.ucla.edu
+ fsusage: port to MacOS X 10.7 with 4 TiB file systems
+ * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
+ implementations that use only 32 bits to count blocks.
+ On typical hosts with 1024-byte blocks, this fails with file
+ systems as small as 4 TiB. Problem reported by Herb Wartens
+ http://debbugs.gnu.org/9140 and this should also fix a similar
+ problem reported by Tim Spriggs http://debbugs.gnu.org/7355.
+
large-inode: New module
* MODULES.html.sh: Add it.
* modules/large-inode, m4/large-inode.m4: New files.
diff --git a/m4/fsusage.m4 b/m4/fsusage.m4
index 3459e63..41c06bc 100644
--- a/m4/fsusage.m4
+++ b/m4/fsusage.m4
@@ -40,7 +40,7 @@ ac_fsusage_space=no
# systems. That system is reported to work fine with STAT_STATFS4 which
# is what it gets when this test fails.
if test $ac_fsusage_space = no; then
- # glibc/{Hurd,kFreeBSD}, MacOS X = 10.4, FreeBSD = 5.0, NetBSD = 3.0,
+ # glibc/{Hurd,kFreeBSD}, FreeBSD = 5.0, NetBSD = 3.0,
# OpenBSD = 4.4, AIX, HP-UX, IRIX, Solaris, Cygwin, Interix, BeOS.
AC_CACHE_CHECK([for statvfs function (SVR4)], [fu_cv_sys_stat_statvfs],
[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include sys/types.h
@@ -56,8 +56,16 @@ a system call.
Do not use Tru64's statvfs implementation
#endif
-#include sys/statvfs.h]],
-[[struct statvfs fsd; statvfs (0,
fsd);]])],
+#include limits.h
+#include sys/statvfs.h
+
+/* Reject implementations, such as MacOS X 10.7, where f_blocks is a
+ 32-bit quantity; that commonly limits file systems to 4 TiB, a
+ ridiculously small limit these days. */
+struct statvfs fsd;
+int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1];
+]],
+[[statvfs (0, fsd);]])],
[fu_cv_sys_stat_statvfs=yes],
[fu_cv_sys_stat_statvfs=no])])
if test $fu_cv_sys_stat_statvfs = yes; then
@@ -94,8 +102,8 @@ if test $ac_fsusage_space = no; then
fi
if test $ac_fsusage_space = no; then
- # glibc/Linux, MacOS X 10.4, FreeBSD 5.0, NetBSD 3.0, OpenBSD 4.4.
- # (glibc/{Hurd,kFreeBSD}, MacOS X = 10.4, FreeBSD = 5.0, NetBSD = 3.0,
+ # glibc/Linux, MacOS X, FreeBSD 5.0, NetBSD 3.0, OpenBSD 4.4.
+ # (glibc/{Hurd,kFreeBSD}, FreeBSD = 5.0, NetBSD = 3.0,
# OpenBSD = 4.4, AIX, HP-UX, OSF/1, Cygwin already handled above.)
# (On IRIX you need to include sys/statfs.h, not only sys/mount.h and
# sys/vfs.h.)
--
1.7.4.4
bug#9140: [PATCH 2/3] large-inode: New module
On 22/07/11 18:22, Paul Eggert wrote: Here's another module I'm adding to coreutils. I plan to change coreutils to use it. Another possibility, I suppose, would be to fold it into AC_SYS_LARGEFILE. Herb, can you please try out the effect of this module by appending #define _DARWIN_USE_64_BIT_INODE 1 to lib/config.h after running 'configure'? Thanks. If it works it would be better in AC_SYS_LARGEFILE (as well) I think. Though the original reporter of this issue said it didn't work. Though I now notice I asked him to put it at the top of df.c rather than somewhere significant to fsusage.c cheers, Pádraig.
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Paul Eggert wrote: Thanks for the further information about the problem with MacOS X 10.7. Can you please try this patch? If you don't have autotools installed, you can edit 'configure' by hand and install a similar patch there. Thanks. I have pushed this into gnulib as it seems pretty simple. The basic idea is to avoid statvfs on MacOS X because its block counts can't go above 2**31. Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. On typical hosts with 1024-byte blocks, this fails with file systems as small as 4 TiB. Problem reported by Herb Wartens http://debbugs.gnu.org/9140 and this should also fix a similar problem reported by Tim Spriggs http://debbugs.gnu.org/7355. ... +/* Reject implementations, such as MacOS X 10.7, where f_blocks is a + 32-bit quantity; that commonly limits file systems to 4 TiB, a + ridiculously small limit these days. */ +struct statvfs fsd; +int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1]; +]], Nice. That looks like it will do the job, and since all statvfs implementations have the f_blocks member, it should be safe. Thanks.
bug#9141: Coreutils Compiler Warnings on OSX 10.7 (Lion)
On 07/21/11 16:00, Herb Wartens wrote: I am still seeing the fdatasync warnings though. I just pushed this patch, which should fix things. Please let me know if it doesn't. Looking around the net, it appears that the fdatasync of Mac OS 10.7 is not effective, so it's better to be safe here and act as if it doesn't exist. From 339613b6bdd8d4a51435cba9e903b8b5bb082da4 Mon Sep 17 00:00:00 2001 From: Paul Eggert egg...@cs.ucla.edu Date: Fri, 22 Jul 2011 13:40:03 -0700 Subject: [PATCH] dd, shred: use fdatasync only if declared * m4/jm-macros.m4 (coreutils_MACROS): Use fdatasync only if declared. MacOS X 10.7 has an fdatasync that is not declared, and is rumored to be ineffective. (Bug#9141) --- m4/jm-macros.m4 | 14 ++ 1 files changed, 10 insertions(+), 4 deletions(-) diff --git a/m4/jm-macros.m4 b/m4/jm-macros.m4 index 6e97221..58b000d 100644 --- a/m4/jm-macros.m4 +++ b/m4/jm-macros.m4 @@ -93,14 +93,20 @@ AC_DEFUN([coreutils_MACROS], cu_PREREQ_STAT_PROG # for dd.c and shred.c - coreutils_saved_libs=$LIBS -LIB_FDATASYNC= + # + # Use fdatasync only if declared. On MacOS X 10.7, fdatasync exists but + # is not declared, and is ineffective. + LIB_FDATASYNC= + AC_SUBST([LIB_FDATASYNC]) + AC_CHECK_DECLS_ONCE([fdatasync]) + if test $ac_cv_have_decl_fdatasync = yes; then +coreutils_saved_libs=$LIBS AC_SEARCH_LIBS([fdatasync], [rt posix4], [test $ac_cv_search_fdatasync = none required || LIB_FDATASYNC=$ac_cv_search_fdatasync]) -AC_SUBST([LIB_FDATASYNC]) AC_CHECK_FUNCS([fdatasync]) - LIBS=$coreutils_saved_libs +LIBS=$coreutils_saved_libs + fi # Check whether libcap is usable -- for ls --color support LIB_CAP= -- 1.7.4.4
bug#9141: fdatasync module proposal
Surely coreutils is not the only program that will have problems
with fdatasync on Mac OS. How about the following gnulib patches?
One is for fdatasync, the other for its tests.
From d5c4b4b9266db16c8477a94640290e0b662cee93 Mon Sep 17 00:00:00 2001
From: Paul Eggert egg...@cs.ucla.edu
Date: Fri, 22 Jul 2011 14:00:22 -0700
Subject: [PATCH 1/2] fdatasync: New module
* MODULES.html.sh: Add it.
* doc/posix-functions/fdatasync.texi (fdatasync): Document it.
* lib/unistd.in.h (fdatasync): Arrange for replacement.
* m4/unistd_h.m4 (gl_UNISTD_H): Prepare for warn-on-use.
(gl_UNISTD_H_DEFAULTS): Define defaults for it.
* modules/unistd (unistd.h): Substitute for its vars.
* lib/fdatasync.c, m4/fdatasync.m4, modules/fdatasync: New files.
---
ChangeLog |9 +
MODULES.html.sh|1 +
doc/posix-functions/fdatasync.texi | 11 +++
lib/fdatasync.c| 30 ++
lib/unistd.in.h| 28
m4/fdatasync.m4| 34 ++
m4/unistd_h.m4 |6 +-
modules/fdatasync | 32
modules/unistd |4
9 files changed, 150 insertions(+), 5 deletions(-)
create mode 100644 lib/fdatasync.c
create mode 100644 m4/fdatasync.m4
create mode 100644 modules/fdatasync
diff --git a/ChangeLog b/ChangeLog
index b652f25..15b0fe5 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,14 @@
2011-07-22 Paul Eggert egg...@cs.ucla.edu
+ fdatasync: New module
+ * MODULES.html.sh: Add it.
+ * doc/posix-functions/fdatasync.texi (fdatasync): Document it.
+ * lib/unistd.in.h (fdatasync): Arrange for replacement.
+ * m4/unistd_h.m4 (gl_UNISTD_H): Prepare for warn-on-use.
+ (gl_UNISTD_H_DEFAULTS): Define defaults for it.
+ * modules/unistd (unistd.h): Substitute for its vars.
+ * lib/fdatasync.c, m4/fdatasync.m4, modules/fdatasync: New files.
+
fsusage: port to MacOS X 10.7 with 4 TiB file systems
* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
implementations that use only 32 bits to count blocks.
diff --git a/MODULES.html.sh b/MODULES.html.sh
index 80befa9..a185c42 100755
--- a/MODULES.html.sh
+++ b/MODULES.html.sh
@@ -2360,6 +2360,7 @@ func_all_modules ()
func_module fclose
func_module fcntl-h
func_module fcntl
+ func_module fdatasync
func_module flock
func_module fopen
func_module fprintf-posix
diff --git a/doc/posix-functions/fdatasync.texi
b/doc/posix-functions/fdatasync.texi
index 8631207..2c581d8 100644
--- a/doc/posix-functions/fdatasync.texi
+++ b/doc/posix-functions/fdatasync.texi
@@ -4,15 +4,18 @@
POSIX specification:@*
@url{http://www.opengroup.org/onlinepubs/9699919799/functions/fdatasync.html}
-Gnulib module: ---
+Gnulib module: fdatasync
Portability problems fixed by Gnulib:
@itemize
+@item
+This function is present but undeclared and ineffective on some platforms:
+MacOS X 10.7
+@item
+This function is missing on some platforms:
+MacOS X 10.5, FreeBSD 6.0, OpenBSD 3.8, IRIX 5.3, Solaris 2.4, mingw, Interix
3.5, BeOS.
@end itemize
Portability problems not fixed by Gnulib:
@itemize
-@item
-This function is missing on some platforms:
-MacOS X 10.5, FreeBSD 6.0, OpenBSD 3.8, IRIX 5.3, Solaris 2.4, mingw, Interix
3.5, BeOS.
@end itemize
diff --git a/lib/fdatasync.c b/lib/fdatasync.c
new file mode 100644
index 000..fbb7b5a
--- /dev/null
+++ b/lib/fdatasync.c
@@ -0,0 +1,30 @@
+/* Emulate fdatasync on platforms that lack it.
+
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see http://www.gnu.org/licenses/. */
+
+/* written by Paul Eggert */
+
+#include config.h
+
+#include unistd.h
+
+int
+fdatasync (int fd)
+#undef fdatasync
+{
+ /* This isn't right, but it's the best we can do portably. */
+ return fsync (fd);
+}
diff --git a/lib/unistd.in.h b/lib/unistd.in.h
index e612fb3..aac5bb2 100644
--- a/lib/unistd.in.h
+++ b/lib/unistd.in.h
@@ -483,6 +483,34 @@ _GL_WARN_ON_USE (fchownat, fchownat is not portable -
#endif
+#if @GNULIB_FDATASYNC@
+/* Synchronize data changes to a file.
+ Return 0 if successful, otherwise -1 and errno set.
+ See POSIX:2001 specification
+
bug#8887: [PATCH 1/2] tests: make test runner a script, not a shell function
Jim Meyering wrote: Stefano Lattarini wrote: Subject: [PATCH 2/2] tests: avoid extra forks in the testsuite * tests/shell-or-perl: Prefer the `read' builtin over `grep' to look at the shebang line of test scripts. Since `read' is a special builtin, it might abort the whole program upon failures, so add extra sanity checks, verifying that the test script exists and is readable, before trying to read from it. ... Thank you. I pushed that as-is. And am marking this as done.
bug#8848: [PATCH] tests: avoid sort-spinlock-abuse false positive under heavy load
Jim Meyering wrote: When building with -j25 on a 6/12-core system, this test would fail about half of the time. With the patch below, it passed make -j25 check 30 times in a row. From d40c2045707bad96e7a8caff2283b537163b8919 Mon Sep 17 00:00:00 2001 From: Jim Meyering meyer...@redhat.com Date: Mon, 13 Jun 2011 11:05:49 +0200 Subject: [PATCH] tests: avoid sort-spinlock-abuse false positive under heavy load * tests/misc/sort-spinlock-abuse: This test would frequently fail when run on a system under heavy load. Increase duration and limit. --- tests/misc/sort-spinlock-abuse |9 + 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/misc/sort-spinlock-abuse b/tests/misc/sort-spinlock-abuse index b5ca30f..fc9612c 100755 --- a/tests/misc/sort-spinlock-abuse +++ b/tests/misc/sort-spinlock-abuse @@ -26,13 +26,14 @@ grep '^#define HAVE_PTHREAD_T 1' $CONFIG_HEADER /dev/null || seq 10 in || framework_failure_ mkfifo_or_skip_ fifo -# Arrange for sort to require 5.0+ seconds of wall-clock time, +# Arrange for sort to require 8.0+ seconds of wall-clock time, This was pushed, and subsequently the entire test was mostly-disabled as very expensive, so I'm closing this.
bug#7992: cut segmentation fault with unbounded ranges
Jim Meyering wrote: Paul Marinescu wrote: In coreutils 8.9 (latest), the following commands trigger an invalid memory access. cut -c1234567890- --output-d=: foo cut -f1234567890- --output-d=: foo cut -b1234567890- --output-d=: foo The number 1234567890 is just a random number 'big enough' to make the invalid access generate a segmentation fault but the invalid access happens for values as low as 8 (valgrind) The problem is that ranges going to end of line (i.e., 'x-') are not taken into account when calculating the size of the printable_field vector, but their lower bound is used as an index on line 525: if (output_delimiter_specified !complement eol_range_start !is_printable_field (eol_range_start)) Thanks a lot for the report. Here's a fix: ... Subject: [PATCH] cut: don't segfault for large unbounded range * src/cut.c (set_fields): When computing the maximum range endpoint, take into consideration the start of any unbounded range, like 999-. * NEWS (Bug fixes): Mention it. * tests/misc/cut (big-unbounded-b,c,f): Add tests. Reported by Paul Marinescu in http://debbugs.gnu.org/7993 The bug was introduced on 2004-12-04 via commit 7380cf79. ... * Noteworthy changes in release ?.? (-??-??) [?] +** Bug fixes + + cut could segfault when invoked with a user-specified output + delimiter and an unbounded range like -f1234567890-. + [bug introduced in coreutils-5.3.0] + Fixed, so closing.
bug#8021: bootstrap failed on solaris10
Jim Meyering wrote: Jeff liu wrote: I was ran into a wired issue when running `bootstrap` on solaris10 with bash-3.00, bash-3.00# ./bootstrap ./bootstrap: syntax error at line 328: `p1=$' unexpected Try bash ./bootstrap. Otherwise, bootstrap uses /bin/sh, which (on that system) doesn't know about the $(...) syntax. bash-3.00# uname -a SunOS pibroch 5.10 Generic_142910-17 i86pc i386 i86pc Coreutils is cloned from upstream, I am intend to try it on Solaris for sparse file copy through lseek. Jeff confirmed that using bash worked around the problem, so I'm closing this.
bug#8103: tac RFE: accept -z, --zero-terminated option (was: bug#8103: NUL terminated lines
retitle 8103 tac RFE: accept -z, --zero-terminated option tags 8103 + notabug thanks Jim Meyering wrote: Bjartur Thorlacius wrote: Maybe we should modify tac to add the -z option. Would you care to write a patch? It would be redundant, as tac -s $'\0' is equivalent. Are you using a non-GNU version of tac? If so, please tell us which one -- that may influence the decision of whether to make -s '' work or to add -z. With GNU tac, that has never worked: $ tac -s '' tac: separator cannot be empty Making -s accommodate an empty string argument is a possibility, but that change looks like it'd be relatively disruptive. Unless there is already an implementation for which -s '' works, I like the idea of adding -z. I've changed the title and marking this as not a bug.
bug#7976: Sitspace install error
tags 7976 + notabug thanks Turner Toure D NGA-PTS USA CTR wrote: I'm getting this error on step 7, invalid configuration x86_64-unknown-linux'. I'm running centos 5.5. Please assist. I'm assuming I need an updated config.guess, but I may be wrong.
bug#7965: file util doesn't note CRLF in troff input
tags 7965 + notabug close 7965 thanks Doug McIlroy wrote: When file-5.03 and file-5.04 identify troff or preprocessor input that contains CRLF line terminators, they fail to report the CRLF property. While the distinction doesn't matter to the groff suite, it does to other utilities. It is wrong to hide it. This is not a coreutils bug.
bug#9140: [PATCH 2/3] large-inode: New module
On 07/22/11 10:45, Pádraig Brady wrote: If it works it would be better in AC_SYS_LARGEFILE (as well) I think. OK, thanks, I pushed this into gnulib. Assuming it works (and I don't see why it'd hurt) I'll push it into autoconf. From a7df7c7d3f2f32d0cad31f65505cfee939d91f4a Mon Sep 17 00:00:00 2001 From: Paul Eggert egg...@cs.ucla.edu Date: Fri, 22 Jul 2011 15:06:36 -0700 Subject: [PATCH] largefile: new module, replacing large-inode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pádraig Brady suggested this in http://debbugs.gnu.org/9140#20. * MODULES.html.sh: Add largefile, remove large-inode. * modules/largefile, m4/largefile.m4: New files. * modules/large-inode, m4/large-inode.m4: Remove. --- ChangeLog |6 +++ MODULES.html.sh |2 +- m4/large-inode.m4 | 24 m4/largefile.m4 | 104 +++ modules/large-inode | 22 --- modules/largefile | 22 +++ 6 files changed, 133 insertions(+), 47 deletions(-) delete mode 100644 m4/large-inode.m4 create mode 100644 m4/largefile.m4 delete mode 100644 modules/large-inode create mode 100644 modules/largefile diff --git a/ChangeLog b/ChangeLog index b652f25..c44292e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,11 @@ 2011-07-22 Paul Eggert egg...@cs.ucla.edu + largefile: new module, replacing large-inode + Pádraig Brady suggested this in http://debbugs.gnu.org/9140#20. + * MODULES.html.sh: Add largefile, remove large-inode. + * modules/largefile, m4/largefile.m4: New files. + * modules/large-inode, m4/large-inode.m4: Remove. + fsusage: port to MacOS X 10.7 with 4 TiB file systems * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. diff --git a/MODULES.html.sh b/MODULES.html.sh index 80befa9..6d84587 100755 --- a/MODULES.html.sh +++ b/MODULES.html.sh @@ -2608,6 +2608,7 @@ func_all_modules () func_module filenamecat-lgpl func_module fts func_module isdir + func_module largefile func_module lchmod func_module lchown func_module mkancesdirs @@ -2656,7 +2657,6 @@ func_all_modules () func_module file-set func_module hash-triple func_module i-ring - func_module large-inode func_module same-inode func_end_table diff --git a/m4/large-inode.m4 b/m4/large-inode.m4 deleted file mode 100644 index b41a391..000 --- a/m4/large-inode.m4 +++ /dev/null @@ -1,24 +0,0 @@ -# Enable large inode numbers on systems normally without them. -*- Autoconf -*- - -# Copyright (C) 2011 Free Software Foundation, Inc. -# This file is free software; the Free Software Foundation -# gives unlimited permission to copy and/or distribute it, -# with or without modifications, as long as this notice is preserved. - -# written by Paul Eggert - -AC_DEFUN([gl_SYS_LARGE_INODE], -[ - dnl Many systems enable large inodes if you enable large offsets. - AC_REQUIRE([AC_SYS_LARGEFILE]) - - dnl Some Mac OS X variants won't access large inode numbers by default. - dnl Defining _DARWIN_USE_64_BIT_INODE fixes this. See - dnl http://developer.apple.com/library/mac/releasenotes/Darwin/SymbolVariantsRelNotes. - dnl - dnl The simplest thing is to define this symbol everywhere. - dnl That helps on the affected systems, and doesn't hurt anywhere. - AC_DEFINE([_DARWIN_USE_64_BIT_INODE], [1], -[Define if you want to use large inode numbers - when running on Mac OS X 10.5 or later.]) -]) diff --git a/m4/largefile.m4 b/m4/largefile.m4 new file mode 100644 index 000..6986244 --- /dev/null +++ b/m4/largefile.m4 @@ -0,0 +1,104 @@ +# Enable large files on systems where this is not the default. + +# Copyright 1992-1996, 1998-2011 Free Software Foundation, Inc. +# This file is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# The following implementation works around a problem in autoconf = 2.68; +# AC_SYS_LARGEFILE does not configure for large inodes on Mac OS X 10.5. +m4_version_prereq([2.69], [] ,[ + +# _AC_SYS_LARGEFILE_TEST_INCLUDES +# --- +m4_define([_AC_SYS_LARGEFILE_TEST_INCLUDES], +[@%:@include sys/types.h + /* Check that off_t can represent 2**63 - 1 correctly. +We can't simply define LARGE_OFF_T to be 9223372036854775807, +since some C++ compilers masquerading as C compilers +incorrectly reject 9223372036854775807. */ +@%:@define LARGE_OFF_T (((off_t) 1 62) - 1 + ((off_t) 1 62)) + int off_t_is_large[[(LARGE_OFF_T % 2147483629 == 721 + LARGE_OFF_T % 2147483647 == 1) + ? 1 : -1]];[]dnl +]) + + +# _AC_SYS_LARGEFILE_MACRO_VALUE(C-MACRO, VALUE, +# CACHE-VAR, +# DESCRIPTION, +# PROLOGUE,
bug#7085: fmt (GNU coreutils) 6.10
tags 7085 + notabug close 7085 thanks Denis M. Wilson wrote: This program does not deal properly with CRLF terminators. Gratuitous CRs are left in joined lines; they should be removed. The user may want to keep the CRLF style or change to Unix (LF). There should be an option for this. As seen in the rest of this thread, this is not a bug.
bug#7061: loop on touch (fdutimens)
tags 7061 + notabug close 7061 thanks C de-Avillez wrote: ... Indeed, building from a fresh GIT did the trick. I am guessing I something got left over from previous builds... This bug can be closed. Done.
bug#7424: Tail -n doesn't work on Red-Hat Linux 5.5
tags 7424 + notabug close 7424 thanks Eric Blake wrote: [as another netiquette tip: please don't top-post on technical lists] On 11/17/2010 12:03 PM, Parsons, Larry wrote: Eric, All, I just went to another box and entered in the command that you said shouldn't work. It works just the way you said it shouldn't. Please, so that we can have some idea of what you are seeing, include an example of what you typed, what happened, and what you wanted to have happen. You _still_ haven't told us what you think is a bug, so we _still_ don't have any idea how to help you, other than to point you to FAQ that may or may not be relevant to your situation. We do appreciate the time you've taken to try and report something, but there are some steps you could take to make your request go a lot further: http://www.catb.org/~esr/faqs/smart-questions.html http://www.chiark.greenend.org.uk/~sgtatham/bugs.html Here's an example report that would qualify as more informative: I reduced things down to this simple testcase: $ printf 'a\nb\n' | tail -1 b but since I didn't type the POSIX 2008 spelling of: tail -n -1 I was surprised that it did the same thing by printing the last line instead of being a syntax error and I didn't realize it was a public forum. Most bug reporting emails are publicly archived; and it's always safer to start from that assumption. However I did copy everyone you did. Ah, but my email didn't have an unenforceable disclaimer tacked on by my employer. Please feel free to repond and teach me about netiquette while you just lack any kind of etiquette. My original reply was terse to match the terseness of your original report. I'm sorry if my reply unintentionally came across as harsh or demeaning; that certainly was not my intent, and the text-only medium of email lacks emotion and tone of voice that convey the rest of the meaning behind a spoken message. Whether or not I'm always successful at my attempts to be polite may be another matter, but you may feel free to take that up with me off-list if you have suggestions for how I could improve in my interpersonal skills. Due to the lack of actual details, I assume this wasn't a real problem. Larry, if you can provide evidence of a problem, please simply reply and we'll investigate. In the mean time, I'm closing this as not a bug.
bug#7483: Tail claims bad file descriptor when following
tags 7483 + notabug close 7483 thanks Paul Eggert wrote: On 11/26/2010 12:44 AM, Johan Pretorius wrote: I use tail 5.3.0 in Windows 7. Hmm, that's rather old. I suggest upgrading to a more-recent version before investigating further. The current version is 8.7. Also, you might try reporting the bug to whoever built it on Windows rather than to bug-coreutils, as it could well be a property of the Windows port rather than a bug in coreutils itself. Thanks for the report. I'm closing this as not a bug. If you find evidence of a bug using the latest, you might want to report it to whomever wrote packaged coreutils for windows 7.
bug#9141: [PATCH 1/3] extensions: Enable extensions on MacOS X 10.5 and later.
Paul Eggert wrote: Recent versions of MacOS seem to have a _DARWIN_C_SOURCE flag that has roughly the same role that _GNU_SOURCE has for GNU systems. It's not roughly the same. _GNU_SOURCE makes some symbols visible that are not visible by default. Whereas _DARWIN_C_SOURCE makes some symbols visible that are already visible by default but hidden when _POSIX_C_SOURCE is defined. The usual idiom in the MacOS X header files is #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) So, I expect that your change will be a no-op for almost everyone. Bruno -- In memoriam Dmitry Pavlov http://en.wikipedia.org/wiki/Dmitry_Pavlov_(general)
bug#9141: fdatasync module proposal
Hi Paul,
Surely coreutils is not the only program that will have problems
with fdatasync on Mac OS. How about the following gnulib patches?
One is for fdatasync, the other for its tests.
Looks mostly good. Just small comments:
--- a/lib/unistd.in.h
+++ b/lib/unistd.in.h
@@ -483,6 +483,34 @@ _GL_WARN_ON_USE (fchownat, fchownat is not portable -
#endif
+#if @GNULIB_FDATASYNC@
+/* Synchronize data changes to a file.
+ Return 0 if successful, otherwise -1 and errno set.
+ See POSIX:2001 specification
+ http://www.opengroup.org/susv3xsh/fdatasync.html. */
POSIX:2001 is superseded by POSIX:2008. Even if the wording in both standards
is the same for this function, it is good to encourage people to look at and
to refer to the newest standard. So, here I would write
See POSIX:2008 specification
http://www.opengroup.org/onlinepubs/9699919799/functions/fdatasync.html.
*/
diff --git a/m4/fdatasync.m4 b/m4/fdatasync.m4
new file mode 100644
index 000..af17970
--- /dev/null
+++ b/m4/fdatasync.m4
@@ -0,0 +1,34 @@
...
+ else
+gl_saved_libs=$LIBS
+ AC_SEARCH_LIBS([fdatasync], [rt posix4],
+ [test $ac_cv_search_fdatasync = none required ||
+ LIB_FDATASYNC=$ac_cv_search_fdatasync])
+ AC_CHECK_FUNCS([fdatasync])
+LIBS=$gl_saved_libs
Here one could add a comment like:
dnl Solaris = 2.6 has fdatasync() in libposix4.
dnl Solaris 7..10 has it in librt.
Just for reference, because in 5 years nobody would remember.
diff --git a/modules/fdatasync b/modules/fdatasync
new file mode 100644
index 000..94980ec
--- /dev/null
+++ b/modules/fdatasync
...
+fsync [test $HAVE_FDATASYNC = 0]
+unistd
+
+configure.ac:
+gl_FUNC_FDATASYNC
+if test $HAVE_FDATASYNC = 0; then
It is safer (w.r.t. future modifications) and more consistent with the
hundreds of other modules to also test $REPLACE_FDATASYNC here:
[test $HAVE_FDATASYNC = 0 || test $REPLACE_FDATASYNC = 1]
const char *file = test-fsync.txt;
With this definition, the test-fsync and test-fdatasync programs will write
to the same file. That is, when run with make -j2, they may collide.
You need to take a different file name for test-fdatasync.
--- /dev/null
+++ b/tests/test-fdatasync.c
@@ -0,0 +1,2 @@
+#define FSYNC fdatasync
+#include test-fsync.c
diff --git a/tests/test-fsync.c b/tests/test-fsync.c
index 2627d0c..6bac01c 100644
--- a/tests/test-fsync.c
+++ b/tests/test-fsync.c
@@ -18,8 +18,12 @@
#include unistd.h
+#ifndef FSYNC
+# define FSYNC fsync
+#endif
+
#include signature.h
-SIGNATURE_CHECK (fsync, int, (int));
+SIGNATURE_CHECK (FSYNC, int, (int));
#include errno.h
#include fcntl.h
@@ -32,7 +36,7 @@ main (void)
int fd;
const char *file = test-fsync.txt;
- if (fsync (0) != 0)
+ if (FSYNC (0) != 0)
{
ASSERT (errno == EINVAL /* POSIX */
|| errno == ENOTSUP /* seen on MacOS X 10.5 */
@@ -42,7 +46,7 @@ main (void)
fd = open (file, O_WRONLY|O_CREAT|O_TRUNC, 0644);
ASSERT (0 = fd);
ASSERT (write (fd, hello, 5) == 5);
- ASSERT (fsync (fd) == 0);
+ ASSERT (FSYNC (fd) == 0);
ASSERT (close (fd) == 0);
ASSERT (unlink (file) == 0);
Here, like with test-pselect, I would move everything after the signature test
to a separate file, test-fsync.h, that is included by both test-fsync.c and
test-fdatasync.c. This avoids #ifdefs (following the general rule to prefer
C functions over C macros), and gives freedom to add tests that apply to one
of the functions but not both.
Bruno
--
In memoriam Dmitry Pavlov http://en.wikipedia.org/wiki/Dmitry_Pavlov_(general)
