bug#7572: ping
Ludwig Nussel wrote: Are there any concerns with the patch? It would be really nice to have this merged upstream to avoid further fragmentation. If you now have a complete patch (including documentation, NEWS update and a good ChangeLog), please post it. Or, if you opted not to pursue this, please just close this issue by Cc'ing 7572-d...@debbugs.gnu.org with your reply.
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Paul Eggert wrote: Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. But this change has also effects on other platforms than MacOS X 10.7: - On glibc/Hurd, the type of f_blocks in 'struct statvfs' is __fsblkcnt64_t or __fsblkcnt_t, depending on __USE_FILE_OFFSET64. So fsusage.m4 ought to require AC_SYS_LARGEFILE. - On MacOS X 10.5, the type of f_blocks is fsblkcnt_t, which is 'unsigned int'. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). And 'struct statfs' contains a f_blocks that has 64 bits if and only if __DARWIN_64_BIT_INO_T is defined. So fsusage.m4 ought to require AC_SYS_LARGEFILE, and a module dependency to 'largefile' is needed, so that it picks up your new AC_SYS_LARGEFILE code. - On FreeBSD 6.4, NetBSD 3.0..5.0, OpenBSD 4.9, the type of f_blocks is fsblkcnt_t, which is already a 64-bit integer. So these platforms continue to use statvfs(). - On AIX 5.1..7.1, the type of f_blocks is fsblkcnt_t, which is 'unsigned long', hence usually 32-bit. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'fsblkcnt_t f_blocks' as well, so the switch from statvfs() to statfs() buys us nothing. What would bring something (on AIX = 5.2) is to use statvfs64 or statfs64, which both have a 64-bit f_blocks field. - On HP-UX 11.00..11.31, the type of f_blocks is fsblkcnt_t, which is 64-bit if and only if _APP32_64BIT_OFF_T is defined. By default it is 32-bit, hence the configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has an 'int32_t f_blocks', so the switch from statvfs() to statfs() buys us nothing. What would bring something is to define _FILE_OFFSET_BITS to 64, which has the effect of enabling _APP32_64BIT_OFF_T. Or alternatively, define _LARGEFILE64_SOURCE and use statvfs64 instead of statvfs. - On IRIX 6.5, the type of f_blocks is fsblkcnt_t, which is already a 64-bit integer, so these platforms continue to use statvfs(). One could define _LARGEFILE64_SOURCE and use statvfs64, but that appears to be not necessary. - On Solaris 7..10, the type of f_blocks is fsblkcnt_t, which is 64-bit in 64-bit mode builds (_LP64) or if _FILE_OFFSET_BITS is 64. By default it usually is 32-bit, the configure test therefore now fails, STAT_STATFS4 gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'long f_blocks', so the switch from statvfs() to statfs() buys us nothing. What would bring something is to define _FILE_OFFSET_BITS to 64. - On Cygwin (both 1.5.x and 1.7.5), the type of f_blocks is fsblkcnt_t, which is 'unsigned long', that is, 32-bit. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'long f_blocks', so the switch from statvfs() to statfs() buys us nothing. I don't see an API that supports 64-bit f_blocks on Cygwin. - On Interix 3.5, the type of f_blocks is 'unsigned long', which is 32-bit. The configure test therefore now fails, and I don't know which API gets used instead: Interix has no statfs. So it appears the patch is killing portability to Interix. - On BeOS and Haiku, the type of f_blocks is fsblkcnt_t, which is 'long long', i.e. 64-bit, so these platforms continue to use statvfs(). I will therefore propose 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. 2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. 3) To make use of statvfs64 on AIX. Bruno -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: add large volume support on glibc/Hurd, HP-UX 11, Solaris, MacOS X
1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. Here's the first proposed patch. I verified that - on HP-UX 11.00 and 11.31, Solaris 7 and 10, it causes statvfs() to be used again: checking for statvfs function (SVR4)... no becomes checking for statvfs function (SVR4)... yes Since the check_f_blocks_size is in place, it proves that this causes f_blocks to be 64-bit. - on MacOS X 10.5, it causes fsusage.o to call statfs$INODE64 instead of statfs, thus also supporting large volumes. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Enable large volume support on glibc/Hurd, HP-UX, Solaris, MacOS X. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Require AC_SYS_LARGEFILE. * modules/fsusage (Depends-on): Add largefile. --- m4/fsusage.m4.orig Sat Jul 23 16:25:19 2011 +++ m4/fsusage.m4 Sat Jul 23 16:24:57 2011 @@ -1,4 +1,4 @@ -# serial 28 +# serial 29 # Obtaining file system usage information. # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc. @@ -29,6 +29,12 @@ AC_DEFUN([gl_FILE_SYSTEM_USAGE], [ +dnl Enable large-file support. This has the effect of changing the size +dnl of field f_blocks in 'struct statvfs' from 32 bit to 64 bit on +dnl glibc/Hurd, HP-UX 11, Solaris (32-bit mode). It also changes the size +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on +dnl MacOS X = 10.5 (32-bit mode). +AC_REQUIRE([AC_SYS_LARGEFILE]) AC_MSG_NOTICE([checking how to get file system space usage]) ac_fsusage_space=no --- modules/fsusage.origSat Jul 23 16:25:19 2011 +++ modules/fsusage Sat Jul 23 16:06:17 2011 @@ -7,6 +7,7 @@ m4/fsusage.m4 Depends-on: +largefile stdbool stdint full-read [test $gl_cv_fs_space = yes] -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: revert unintended change on AIX, Cygwin, Interix
2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. Slight correction: It is still needed after 1), at least on MacOS X 10.5. But only on MacOS X. So here's the proposed patch 2. Verified on AIX and Cygwin; I don't have access to an Interix machine. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Restore previous behaviour on AIX, Cygwin, Interix. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Enforce a 64-bit struct statvfs f_blocks field only on MacOS X. *** m4/fsusage.m4.orig Sat Jul 23 16:44:07 2011 --- m4/fsusage.m4 Sat Jul 23 16:42:32 2011 *** *** 62,75 Do not use Tru64's statvfs implementation #endif - #include limits.h #include sys/statvfs.h - /* Reject implementations, such as MacOS X 10.7, where f_blocks is a -32-bit quantity; that commonly limits file systems to 4 TiB, a -ridiculously small limit these days. */ struct statvfs fsd; int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1]; ]], [[statvfs (0, fsd);]])], [fu_cv_sys_stat_statvfs=yes], --- 62,80 Do not use Tru64's statvfs implementation #endif #include sys/statvfs.h struct statvfs fsd; + + #if defined __APPLE__ defined __MACH__ + #include limits.h + /* On MacOS X = 10.5, f_blocks in 'struct statvfs' is a 32-bit quantity; + that commonly limits file systems to 4 TiB. Whereas f_blocks in + 'struct statfs' is a 64-bit type, thanks to the large-file support + that was enabled above. In this case, don't use statvfs(); use statfs() + instead. */ int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1]; + #endif ]], [[statvfs (0, fsd);]])], [fu_cv_sys_stat_statvfs=yes], -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Bruno Haible wrote: Paul Eggert wrote: Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. But this change has also effects on other platforms than MacOS X 10.7: - On glibc/Hurd, the type of f_blocks in 'struct statvfs' is __fsblkcnt64_t or ... - On MacOS X 10.5, the type of f_blocks is fsblkcnt_t, which is ... - On FreeBSD 6.4, NetBSD 3.0..5.0, OpenBSD 4.9, the type of f_blocks is ... - On AIX 5.1..7.1, the type of f_blocks is fsblkcnt_t, which is 'unsigned long', ... - On HP-UX 11.00..11.31, the type of f_blocks is fsblkcnt_t, which is 64-bit ... - On IRIX 6.5, the type of f_blocks is fsblkcnt_t, which is already a 64-bit ... - On Solaris 7..10, the type of f_blocks is fsblkcnt_t, which is 64-bit in ... - On Cygwin (both 1.5.x and 1.7.5), the type of f_blocks is fsblkcnt_t, which ... - On Interix 3.5, the type of f_blocks is 'unsigned long', which is 32-bit. ... - On BeOS and Haiku, the type of f_blocks is fsblkcnt_t, which is 'long long', ... I will therefore propose 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. 2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. 3) To make use of statvfs64 on AIX. Wow. Thanks for all the analysis and testing.
bug#9140: fsusage: add large volume support on glibc/Hurd, HP-UX 11, Solaris, MacOS X
Bruno Haible wrote: 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. Here's the first proposed patch. I verified that - on HP-UX 11.00 and 11.31, Solaris 7 and 10, it causes statvfs() to be used again: checking for statvfs function (SVR4)... no becomes checking for statvfs function (SVR4)... yes Since the check_f_blocks_size is in place, it proves that this causes f_blocks to be 64-bit. - on MacOS X 10.5, it causes fsusage.o to call statfs$INODE64 instead of statfs, thus also supporting large volumes. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Enable large volume support on glibc/Hurd, HP-UX, Solaris, MacOS X. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Require AC_SYS_LARGEFILE. * modules/fsusage (Depends-on): Add largefile. --- m4/fsusage.m4.origSat Jul 23 16:25:19 2011 +++ m4/fsusage.m4 Sat Jul 23 16:24:57 2011 @@ -1,4 +1,4 @@ -# serial 28 +# serial 29 # Obtaining file system usage information. # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc. @@ -29,6 +29,12 @@ AC_DEFUN([gl_FILE_SYSTEM_USAGE], [ +dnl Enable large-file support. This has the effect of changing the size +dnl of field f_blocks in 'struct statvfs' from 32 bit to 64 bit on +dnl glibc/Hurd, HP-UX 11, Solaris (32-bit mode). It also changes the size +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on +dnl MacOS X = 10.5 (32-bit mode). +AC_REQUIRE([AC_SYS_LARGEFILE]) AC_MSG_NOTICE([checking how to get file system space usage]) ac_fsusage_space=no --- modules/fsusage.orig Sat Jul 23 16:25:19 2011 +++ modules/fsusage Sat Jul 23 16:06:17 2011 @@ -7,6 +7,7 @@ m4/fsusage.m4 Depends-on: +largefile stdbool stdint full-read [test $gl_cv_fs_space = yes] Well-documented, tested, and safe-looking. Thanks!
bug#9140: fsusage: revert unintended change on AIX, Cygwin, Interix
Bruno Haible wrote: 2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. Slight correction: It is still needed after 1), at least on MacOS X 10.5. But only on MacOS X. So here's the proposed patch 2. Verified on AIX and Cygwin; Thanks again. This looks fine. I don't have access to an Interix machine. No problem. It seems to have so few coreutils/gnulib users that it barely qualifies as a reasonable portability target. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Restore previous behaviour on AIX, Cygwin, Interix. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Enforce a 64-bit struct statvfs f_blocks field only on MacOS X. ...
bug#9140: fsusage: add large volume support on AIX
3) To make use of statvfs64 on AIX. Here's the proposed patch for it. As expected we get: - On AIX 5.2, 6.1, 7.1: checking whether to use statvfs64... yes - On AIX 5.1, HP-UX, IRIX, Solaris: checking whether to use statvfs64... no 2011-07-23 Bruno Haible br...@clisp.org fsusage: Enable large volume support on AIX = 5.2. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): If 'struct statvfs64' has a larger f_blocks field than 'struct statvfs', define STAT_STATVFS64 instead of STAT_STATVFS. * lib/fsusage.c (get_fs_usage) [STAT_STATVFS64]: Use statvfs64. --- lib/fsusage.c.orig Sat Jul 23 17:15:05 2011 +++ lib/fsusage.c Sat Jul 23 17:13:38 2011 @@ -23,7 +23,7 @@ #include limits.h #include sys/types.h -#if STAT_STATVFS/* POSIX 1003.1-2001 (and later) with XSI */ +#if STAT_STATVFS || STAT_STATVFS64 /* POSIX 1003.1-2001 (and later) with XSI */ # include sys/statvfs.h #else /* Don't include backward-compatibility files unless they're needed. @@ -106,6 +106,18 @@ ? PROPAGATE_ALL_ONES (fsd.f_frsize) : PROPAGATE_ALL_ONES (fsd.f_bsize)); +#elif defined STAT_STATVFS64/* AIX */ + + struct statvfs64 fsd; + + if (statvfs64 (file, fsd) 0) +return -1; + + /* f_frsize isn't guaranteed to be supported. */ + fsp-fsu_blocksize = (fsd.f_frsize +? PROPAGATE_ALL_ONES (fsd.f_frsize) +: PROPAGATE_ALL_ONES (fsd.f_bsize)); + #elif defined STAT_STATFS2_FS_DATA /* Ultrix */ struct fs_data fsd; @@ -223,7 +235,7 @@ #endif -#if (defined STAT_STATVFS \ +#if (defined STAT_STATVFS || defined STAT_STATVFS64 \ || (!defined STAT_STATFS2_FS_DATA !defined STAT_READ_FILSYS)) fsp-fsu_blocks = PROPAGATE_ALL_ONES (fsd.f_blocks); --- m4/fsusage.m4.orig Sat Jul 23 17:15:05 2011 +++ m4/fsusage.m4 Sat Jul 23 17:12:20 2011 @@ -1,4 +1,4 @@ -# serial 29 +# serial 30 # Obtaining file system usage information. # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc. @@ -81,8 +81,32 @@ [fu_cv_sys_stat_statvfs=no])]) if test $fu_cv_sys_stat_statvfs = yes; then ac_fsusage_space=yes -AC_DEFINE([STAT_STATVFS], [1], - [ Define if there is a function named statvfs. (SVR4)]) +# AIX = 5.2 has statvfs64 that has a wider f_blocks field than statvfs. +# glibc, HP-UX, IRIX, Solaris have statvfs64 as well, but on these systems +# statvfs with large-file support is already equivalent to statvfs64. +AC_CACHE_CHECK([whether to use statvfs64], + [fu_cv_sys_stat_statvfs64], + [AC_LINK_IFELSE( + [AC_LANG_PROGRAM( +[[#include sys/types.h + #include sys/statvfs.h + struct statvfs64 fsd; + int check_f_blocks_larger_in_statvfs64 +[sizeof (((struct statvfs64 *) 0)-f_blocks) + sizeof (((struct statvfs *) 0)-f_blocks) + ? 1 : -1]; +]], +[[statvfs64 (0, fsd);]])], + [fu_cv_sys_stat_statvfs64=yes], + [fu_cv_sys_stat_statvfs64=no]) + ]) +if test $fu_cv_sys_stat_statvfs64 = yes; then + AC_DEFINE([STAT_STATVFS64], [1], +[ Define if statvfs64 should be preferred over statvfs.]) +else + AC_DEFINE([STAT_STATVFS], [1], +[ Define if there is a function named statvfs. (SVR4)]) +fi fi fi -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: add large volume support on AIX
Thanks for all that work in analysis, and porting checking, for statvfs etc. Your patches look good to me too.
bug#9141: [PATCH 1/3] extensions: Enable extensions on MacOS X 10.5 and later.
On 07/22/11 17:23, Bruno Haible wrote: The usual idiom in the MacOS X header files is #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) So, I expect that your change will be a no-op for almost everyone. Thanks for the clarification. I looked through what I could glean from Google searches, and found some places that do not use the above pattern. For example, http://www.opensource.apple.com/source/xnu/xnu-1456.1.26/bsd/sys/select.h says something like this: int pselect(int, fd_set * __restrict, fd_set * __restrict, fd_set * __restrict, const struct timespec * __restrict, const sigset_t * __restrict) #if defined(_DARWIN_C_SOURCE) || defined(_DARWIN_UNLIMITED_SELECT) __DARWIN_EXTSN_C(pselect) #else # if defined(__LP64__) !__DARWIN_NON_CANCELABLE __DARWIN_1050(pselect) # else __DARWIN_ALIAS_C(pselect) # endif #endif ; and http://www.opensource.apple.com/source/Libc/Libc-498.1.7/include/stdlib.h says something like this: #if (__DARWIN_UNIX03 !defined(_POSIX_C_SOURCE)) || defined(_DARWIN_C_SOURCE) || defined(_DARWIN_BETTER_REALPATH) char*realpath(const char * __restrict, char * __restrict) __DARWIN_EXTSN(realpath); #else char*realpath(const char * __restrict, char * __restrict) __DARWIN_ALIAS(realpath); #endif which suggests that realpath and pselect have extended runtime behavior if _DARWIN_C_SOURCE is defined. My guess is that these runtime differences are in the same spirit as _GNU_SOURCE, i.e., they enable desirable changes that POSIX would otherwise prohibit. If so, the patch should be kept. But if these changes are undesirable, the patch should be backed out.
