Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 11:37:54PM +0100, Jeroen van Wolffelaar wrote: If you're going to offer %S to retrieve fundamental blocksize, I think it'd be best to actually really retrieve that value, and do something smart otherwise (error out, noting in the documentation that %S might not always be available?), unless of course it is for a given system actually known that f_bsize is indeed the unit size for free/total blocks. findutils calls this ST_NBLOCKSIZE and does it this way: - /* Extract or fake data from a `struct stat'. ST_BLKSIZE: Preferred I/O blocksize for the file, in bytes. ST_NBLOCKS: Number of blocks in the file, including indirect blocks. ST_NBLOCKSIZE: Size of blocks used when calculating ST_NBLOCKS. */ #ifndef HAVE_STRUCT_STAT_ST_BLOCKS /* ... */ # if defined(hpux) || defined(__hpux__) || defined(__hpux) /* HP-UX counts st_blocks in 1024-byte units. This loses when mixing HP-UX and BSD filesystems with NFS. */ # define ST_NBLOCKSIZE 1024 # else /* !hpux */ # if defined(_AIX) defined(_I386) /* AIX PS/2 counts st_blocks in 4K units. */ # define ST_NBLOCKSIZE (4 * 1024) # endif /* !hpux */ #endif #ifndef ST_NBLOCKSIZE # define ST_NBLOCKSIZE 512 #endif (The actual code is a bit more complex, as it also defines a macro ST_BLKSIZE and defines a more complex value for ST_NBLOCKS on Crays (i.e. UNICOS)). ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
Paul Eggert [EMAIL PROTECTED] wrote: This introduces a new format %S to stat -f -c. ... Jim, this is an incompatible change to the output of stat -f, since it changes the default format to output both block sizes. I think the change is useful, but if you think that's too drastic right now please let me know and I'll back this change out for now. It seems safe enough; I doubt many applications depend on the default output format. Thanks for doing that, and for the clean-up. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
Jeroen van Wolffelaar [EMAIL PROTECTED] writes: I like the (latest) patch from Paul Eggert more, although I'm not sure people will be going to understand the term fundamental block size immediately. They are the name POSIX gave us; there are some advantages to sticking with the standard names. The parenthetical comments (e.g., (for block counts)) attempt to clarify them a bit. I think it's a bit betrayal to the user to just copy f_bsize if f_frsize is unavailable, There's precedent for this in lib/fsusage.c. As I understand it, it doesn't break anything on standard hosts, and it works around some incompatibilities in some nonstandard hosts. If it breaks something on other nonstandard hosts then we can look into the problem at that point. By the way, I didn't yet test the patch, it'd be relatively much work for me, but I'm willing to do so if really nobody else seems able to test this. I can't easily test it on anything like your platform, so it would be useful if you gave it a whirl. You should be able to do this by downloading this: ftp://alpha.gnu.org/gnu/coreutils/coreutils-5.3.0.tar.gz applying the stat.c patch, and running sh configure; make; make check. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 02:42:09PM +0100, Jeroen van Wolffelaar wrote: Package: coreutils Version: 5.2.1-2 %s Optimal transfer block size $ 'Optimal transfer block size', I have the suspection that %s IS the blocksize of the filesystem, which should be used in a number of other parameters in order to translate number of blocks into number of bytes. Is that correct? No.The POSIX standard (in a non-normative section) says :- | The unit for the st_blocks member of the stat structure is not defined | within IEEE Std 1003.1-2001. In some implementations it is 512 | bytes. It may differ on a file system basis. There is no correlation | between values of the st_blocks and st_blksize, and the f_bsize (from | sys/statvfs.h) structure members. | | Traditionally, some implementations defined the multiplier for | st_blocks in sys/param.h as the symbol DEV_BSIZE. I think almost all systems use 512 byte units. James. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 03:39:05PM +, James Youngman wrote: On Tue, Feb 08, 2005 at 02:42:09PM +0100, Jeroen van Wolffelaar wrote: Package: coreutils Version: 5.2.1-2 %s Optimal transfer block size $ 'Optimal transfer block size', I have the suspection that %s IS the blocksize of the filesystem, which should be used in a number of other parameters in order to translate number of blocks into number of bytes. Is that correct? No.The POSIX standard (in a non-normative section) says :- | The unit for the st_blocks member of the stat structure is not defined | within IEEE Std 1003.1-2001. In some implementations it is 512 | bytes. It may differ on a file system basis. There is no correlation | between values of the st_blocks and st_blksize, and the f_bsize (from | sys/statvfs.h) structure members. | | Traditionally, some implementations defined the multiplier for | st_blocks in sys/param.h as the symbol DEV_BSIZE. I think almost all systems use 512 byte units. Ah. On all (GNU/Debian) systems I tried so far though, %s gave 4096, and I needed to multiply free and available block counts (%a and %f) with 4096 to get the actual amount of free diskspace. The two UNIX systems I tried so far neither had GNU stat installed, unfortunately. What I'm interested in to know, is what stat member gives the correct amount of bytes for a block with which I can multiply %f/%a to obtain the amount of free diskspace in bytes, or if that's impossible, some mention of that in a manpage, so that people after me who are going to look for it, will find out easier than I'm doing now. Since the 'df' utility does seem to be able to determine the amount of free diskspace, somehow this would seem to be possible. Thank you for your response, --Jeroen -- Jeroen van Wolffelaar [EMAIL PROTECTED] (also for Jabber MSN; ICQ: 33944357) http://Jeroen.A-Eskwadraat.nl ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 03:39:05PM +, James Youngman wrote:
No.The POSIX standard (in a non-normative section) says :-
| The unit for the st_blocks member of the stat structure is not defined
| within IEEE Std 1003.1-2001. In some implementations it is 512
| bytes. It may differ on a file system basis. There is no correlation
| between values of the st_blocks and st_blksize, and the f_bsize (from
| sys/statvfs.h) structure members.
|
| Traditionally, some implementations defined the multiplier for
| st_blocks in sys/param.h as the symbol DEV_BSIZE.
I think almost all systems use 512 byte units.
I've done a bit more reading and googling, and I found statvfs(2):
struct statvfs {
unsigned long f_bsize;/* file system block size */
unsigned long f_frsize; /* fragment size */
fsblkcnt_t f_blocks; /* size of fs in f_frsize units */
fsblkcnt_t f_bfree;/* # free blocks */
fsblkcnt_t f_bavail; /* # free blocks for non-root */
(...)
and a note in statfs(2):
LSB has deprecated the library calls [f]statfs() and tells us
to use [f]statvfs() instead.
So, since statfs seems deprecated, and statvfs more informative (it
actually carries the units in which free and total size of filesystems
is written), I think it would be useful for stat to support statvfs,
which provides a superset of the functionality of statfs(2).
Thanks,
--Jeroen
--
Jeroen van Wolffelaar
[EMAIL PROTECTED] (also for Jabber MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 03:39:05PM +, James Youngman wrote: No.The POSIX standard (in a non-normative section) says :- | The unit for the st_blocks member of the stat structure is not defined | within IEEE Std 1003.1-2001. In some implementations it is 512 | bytes. It may differ on a file system basis. There is no correlation | between values of the st_blocks and st_blksize, and the f_bsize (from | sys/statvfs.h) structure members. | | Traditionally, some implementations defined the multiplier for | st_blocks in sys/param.h as the symbol DEV_BSIZE. I think almost all systems use 512 byte units. Note where it says some implementations? You're diving heavily into implementation-specific logic, and making unfounded assumptions. The part where it says There is no correlation between values of the st_blocks and st_blksize, and the f_bsize goes with The unit for the st_blocks member of the stat structure is not defined within IEEE Std 1003.1-2001. IOW, POSIX doesn't define any such relationship. A particular implementation *may* define such a relationship (and linux has). On other platforms the st_blocks unit could be constant and the st_blksize might actually be a suggestion to applications as to what the most efficient I/O size is (e.g., the size of a disk stripe). Mike Stone ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
Jeroen van Wolffelaar [EMAIL PROTECTED] wrote:
I've done a bit more reading and googling, and I found statvfs(2):
struct statvfs {
unsigned long f_bsize;/* file system block size */
unsigned long f_frsize; /* fragment size */
fsblkcnt_t f_blocks; /* size of fs in f_frsize units */
fsblkcnt_t f_bfree;/* # free blocks */
fsblkcnt_t f_bavail; /* # free blocks for non-root */
...
So, since statfs seems deprecated, and statvfs more informative (it
actually carries the units in which free and total size of filesystems
is written), I think it would be useful for stat to support statvfs,
which provides a superset of the functionality of statfs(2).
Does stat's --file-system (-f) option do what you want?
$ stat -f /
File: /
ID: 0Namelen: 255 Type: ext2/ext3
Blocks: Total: 1032080Free: 887328 Available: 834894 Size: 4096
Inodes: Total: 525312 Free: 490241
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 05:43:08PM +0100, Jim Meyering wrote: Does stat's --file-system (-f) option do what you want? $ stat -f / File: / ID: 0Namelen: 255 Type: ext2/ext3 Blocks: Total: 1032080Free: 887328 Available: 834894 Size: 4096 No, as strace-ing reveals only statfs is called, and indeed, looking at the source, %s is actually printed there, while it was in this bug previously already discovered that %s is actually NOT the size of a block (%s is f_bsize, and not f_frsize). So, the stat -f / is actually misleading, even though it might be possible that in linux f_bsize and f_frsize are always equal, this is not so defined. --Jeroen -- Jeroen van Wolffelaar [EMAIL PROTECTED] (also for Jabber MSN; ICQ: 33944357) http://Jeroen.A-Eskwadraat.nl ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
Jeroen van Wolffelaar [EMAIL PROTECTED] writes:
See http://bugs.debian.org/294206#msg17 for more information, bottom
line is that statvfs returns a superset of values compared to statfs,
and one of the extra values is f_frsize required for getting the free
disk space in bytes.
OK, thanks, I installed this (revised) patch into coreutils, which I
hope addresses that problem. This supersedes my previous message
on this subject.
This introduces a new format %S to stat -f -c. You need a kernel that
has a working statvfs and f_frsize to notice any difference between %s
and %S. I've tested it on Solaris 8, which has such a kernel. I'd
appreciate your testing it on a Linux libc/kernel that has this
feature (mine doesn't).
Jim, this is an incompatible change to the output of stat -f, since
it changes the default format to output both block sizes. I think the
change is useful, but if you think that's too drastic right now please
let me know and I'll back this change out for now.
2005-02-08 Paul Eggert [EMAIL PROTECTED]
* NEWS: Document stat -f -c %S, plus changes to default formats.
* doc/coreutils.texi (stat invocation): Normalize terminology,
capitalization, and sort order to match --help output. Mention %c
for file systems. Add new -f -c format %S, and document %s versus %S;
problem reported by Jeroen van Wolffelaar.
* src/stat.c (usage): Likewise.
(STATFS_FRSIZE): New macro.
(print_statfs): Use it, for stat -f -c %S.
(do_statfs): Change default formats to output %S.
Index: NEWS
===
RCS file: /fetish/cu/NEWS,v
retrieving revision 1.267
retrieving revision 1.268
diff -p -u -r1.267 -r1.268
--- NEWS15 Jan 2005 19:56:32 - 1.267
+++ NEWS8 Feb 2005 21:37:26 - 1.268
@@ -26,6 +26,11 @@ GNU coreutils NEWS
test now detects integer overflow when evaluating large integers,
rather than silently wrapping around.
+** New features
+
+ stat -f -c %S outputs the fundamental block size (used for block counts).
+ stat -f's default output format has been changed to output this size as well.
+
* Major changes in release 5.3.0 (2005-01-08) [unstable]
** Bug fixes
Index: doc/coreutils.texi
===
RCS file: /fetish/cu/doc/coreutils.texi,v
retrieving revision 1.239
retrieving revision 1.241
diff -p -u -r1.239 -r1.241
--- doc/coreutils.texi 10 Jan 2005 18:12:35 - 1.239
+++ doc/coreutils.texi 8 Feb 2005 21:38:04 - 1.241
@@ -9065,7 +9065,7 @@ Print the information in terse form, sui
@cindex output format
Use @var{format} rather than the default format.
-Interpreted sequences for file stat are:
+The valid format sequences for files are:
@itemize @bullet
@item %a - Access rights in octal
@@ -9074,19 +9074,19 @@ Interpreted sequences for file stat are:
@item %B - The size in bytes of each block reported by @samp{%b}
@item %d - Device number in decimal
@item %D - Device number in hex
[EMAIL PROTECTED] %f - raw mode in hex
[EMAIL PROTECTED] %f - Raw mode in hex
@item %F - File type
[EMAIL PROTECTED] %g - Group Id of owner
[EMAIL PROTECTED] %g - Group ID of owner
@item %G - Group name of owner
@item %h - Number of hard links
@item %i - Inode number
@item %n - File name
[EMAIL PROTECTED] %N - Quoted File name with dereference if symbolic link
[EMAIL PROTECTED] %N - Quoted file name with dereference if symbolic link
@item %o - I/O block size
@item %s - Total size, in bytes
@item %t - Major device type in hex
@item %T - Minor device type in hex
[EMAIL PROTECTED] %u - User Id of owner
[EMAIL PROTECTED] %u - User ID of owner
@item %U - User name of owner
@item %x - Time of last access
@item %X - Time of last access as seconds since Epoch
@@ -9096,19 +9096,21 @@ Interpreted sequences for file stat are:
@item %Z - Time of last change as seconds since Epoch
@end itemize
-Interpreted sequences for file system stat are:
+The valid format sequences for file systems are:
@itemize @bullet
[EMAIL PROTECTED] %n - File name
[EMAIL PROTECTED] %i - File System id in hex
[EMAIL PROTECTED] %a - Free blocks available to non-superuser
[EMAIL PROTECTED] %b - Total data blocks in file system
[EMAIL PROTECTED] %c - Total file nodes in file system
[EMAIL PROTECTED] %d - Free file nodes in file system
[EMAIL PROTECTED] %f - Free blocks in file system
[EMAIL PROTECTED] %i - File System ID in hex
@item %l - Maximum length of file names
[EMAIL PROTECTED] %n - File name
[EMAIL PROTECTED] %s - Block size (for faster transfers)
[EMAIL PROTECTED] %S - Fundamental block size (for block counts)
@item %t - Type in hex
@item %T - Type in human readable form
[EMAIL PROTECTED] %b - Total data blocks in file system
[EMAIL PROTECTED] %f - Free blocks in file system
[EMAIL PROTECTED] %a - Free blocks available to non-superuser
[EMAIL PROTECTED]
Re: Bug#294206: stat(1) unclear about block size
Jeroen van Wolffelaar [EMAIL PROTECTED] writes: On Tue, Feb 08, 2005 at 05:43:08PM +0100, Jim Meyering wrote: Does stat's --file-system (-f) option do what you want? $ stat -f / File: / ID: 0Namelen: 255 Type: ext2/ext3 Blocks: Total: 1032080Free: 887328 Available: 834894 Size: 4096 No, as strace-ing reveals only statfs is called, Which is completely correct. Linux does not have a statvfs syscall, everything is in the C library. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 And now for something completely different. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#294206: stat(1) unclear about block size
On Tue, Feb 08, 2005 at 05:19:37PM -0500, Michael Stone wrote: On Tue, Feb 08, 2005 at 01:56:00PM -0800, Paul Eggert wrote: OK, thanks, I installed this (revised) patch into coreutils, which I hope addresses that problem. No, if you're going to use the statvfs block size you need to pull the other numbers from statvfs also. IOW, just add another flag (-F?) to use statvfs instead of statfs. I don't know of any assurance that they the two sets of numbers will be the same. (And since this entire thread is about the possibility that something might be ambiguous on some implementation somewhere, it doesn't make sense to make an assumption.) Eh, current stat(1) code already uses statvfs if available, so IMHO it would make no sense to add another flag for this, not to mention it'd be a bit weird to propagate library weirdness to a user-level interface like /usr/bin/stat is. If there are differences, they are in current stat already, and they should be fixed one way or the other IMHO. I like the (latest) patch from Paul Eggert more, although I'm not sure people will be going to understand the term fundamental block size immediately. Maybe Unit Block Size (%S) vs. Optimal-I/O blocksize (%s)? Also, I think it's a bit betrayal to the user to just copy f_bsize if f_frsize is unavailable, as who knows that actually that is the correct fundamental blocksize? If it is, then this whole %S vs %s thing isn't needed anyway, the whole point of %S is to have a definite answer to How many bytes is a block as used in the free block count?. If you're going to offer %S to retrieve fundamental blocksize, I think it'd be best to actually really retrieve that value, and do something smart otherwise (error out, noting in the documentation that %S might not always be available?), unless of course it is for a given system actually known that f_bsize is indeed the unit size for free/total blocks. By the way, I didn't yet test the patch, it'd be relatively much work for me, but I'm willing to do so if really nobody else seems able to test this. Thank you all, --Jeroen -- Jeroen van Wolffelaar [EMAIL PROTECTED] (also for Jabber MSN; ICQ: 33944357) http://Jeroen.A-Eskwadraat.nl ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
