Re: making a file sparse - in-place?
On 01/24/2014 03:56 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 03:30:20AM +, Pádraig Brady wrote: On 01/24/2014 03:12 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 02:59:41AM +, Pádraig Brady wrote: On 01/24/2014 02:41 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 01:07:21AM +, Pádraig Brady wrote: On 01/24/2014 12:47 AM, Bernhard Voelker wrote: Inspired by a recent post on util-linux ML [1], talking about turning a file into a sparse file in-place, i.e. not using a 2-step approach like `cp --sparse file file2 mv file2 file`), I thought, hey, don't we have this in coreutils already? But, a question about policy: is it okay to implement linux-only extensions here ? If the current system doesn't support in place sparsify, then be could document that limitation along the same lines as the conv=notrunc case above. Sorry, not sure I follow you here. If we have the same documentation, the conv=notrunc case never says anything platform nor filesystem specific. And that is a big difference, IMHO. And, also, truncate() is a quite common function. I meant there are cases where the current conv=sparse code doesn't create a sparse file when the dest is opened with conv=notrunc. We documented that, which is was I pasted here. Similarly we could document caveats with in place sparsification. I mean, is there any linux specific extension to a tool in coreutils ? (optimization underneath, using linux-only features is totally different IMHO). In place punch hole may not work on all file systems anyway. So the system specific point is moot here. We would always have to handle cases where it was not supported. But it's just a matter of policy. Do you know what the policy is ? We use system specific functionality where appropriate. We would just decide what to do when the current file doesn't support punch hole (which we have to do anyway as said above). Do we fail or ignore. Generally sparse requests are advisory, and indeed in dd we already quietly disable sparse mode if not supported. So we would do the same if punch hole was not supported on the dest. If one wanted more portable guarantees about sparsifying a file, then it would be best to use a temporary file anyway. ¿? What do you mean by this ? Doing it that way would work in more situations If there are other methods to punch a hole in a file on other systems, they can be added as an option to coreutils without changing the interface. AFAIK, there aren't. And, as I said, is not linux specific only, it also is fs specific. There is fcntl(F_FREESP) on solaris, with mention of porting that to FREEBSD. If I have to guess, this flag is like other useful flags linux adds and most other platforms don't. So you are exposing a feature here, that will only will work on linux with certain filesystems. Probably there is a policy already esablished for this ? (I mean, if it's ok or not) If there is a sensible fall back, then it's OK to use non ubiquitous interfaces. thanks, Pádraig.
Re: making a file sparse - in-place?
On Fri, Jan 24, 2014 at 12:49:18PM +, Pádraig Brady wrote: On 01/24/2014 03:56 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 03:30:20AM +, Pádraig Brady wrote: On 01/24/2014 03:12 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 02:59:41AM +, Pádraig Brady wrote: On 01/24/2014 02:41 AM, Rodrigo Campos wrote: On Fri, Jan 24, 2014 at 01:07:21AM +, Pádraig Brady wrote: On 01/24/2014 12:47 AM, Bernhard Voelker wrote: Inspired by a recent post on util-linux ML [1], talking about turning a file into a sparse file in-place, i.e. not using a 2-step approach like `cp --sparse file file2 mv file2 file`), I thought, hey, don't we have this in coreutils already? But it's just a matter of policy. Do you know what the policy is ? We use system specific functionality where appropriate. We would just decide what to do when the current file doesn't support punch hole (which we have to do anyway as said above). Do we fail or ignore. Generally sparse requests are advisory, and indeed in dd we already quietly disable sparse mode if not supported. So we would do the same if punch hole was not supported on the dest. Ohh, okay! If there are other methods to punch a hole in a file on other systems, they can be added as an option to coreutils without changing the interface. AFAIK, there aren't. And, as I said, is not linux specific only, it also is fs specific. There is fcntl(F_FREESP) on solaris, with mention of porting that to FREEBSD. Ohh, nice. I didn't know about them! If I have to guess, this flag is like other useful flags linux adds and most other platforms don't. So you are exposing a feature here, that will only will work on linux with certain filesystems. Probably there is a policy already esablished for this ? (I mean, if it's ok or not) If there is a sensible fall back, then it's OK to use non ubiquitous interfaces. Great! Thanks a lot, and sorry for all the emails :) Rodrigo
script suggestion: 'check_program' to easily run multiple tests
Hello,
Attached is a small script I've been using.
It helps running multiple tests for a given program.
example:
./scripts/check_program sort
Will find all sort-related tests (based on filename) and run them.
Adding -e or -v also runs expensive and very expensive tests:
examle:
./scripts/check_program -v sort
is equivalent to:
make check VERBOSE=yes SUBDIRS=. \
RUN_EXPENSIVE_TESTS=yes \ RUN_VERY_EXPENSIVE_TESTS=yes \
TESTS=./tests/misc/sort-NaN-infloop.sh
./tests/misc/sort-benchmark-random.sh
./tests/misc/sort-compress-hang.sh
./tests/misc/sort-compress-proc.sh
./tests/misc/sort-compress.sh
./tests/misc/sort-continue.sh
./tests/misc/sort-debug-keys.sh
./tests/misc/sort-debug-warn.sh
./tests/misc/sort-discrim.sh
./tests/misc/sort-exit-early.sh
./tests/misc/sort-files0-from.pl
./tests/misc/sort-float.sh
./tests/misc/sort-merge-fdlimit.sh
./tests/misc/sort-merge.pl
./tests/misc/sort-month.sh
./tests/misc/sort-rand.sh
./tests/misc/sort-spinlock-abuse.sh
./tests/misc/sort-stale-thread-mem.sh
./tests/misc/sort-u-FMR.sh
./tests/misc/sort-unique-segv.sh
./tests/misc/sort-unique.sh
./tests/misc/sort-version.sh
./tests/misc/sort.pl
If others find it useful, you're welcomed to add this.
-gordon
From 965a01bfaf129b4d1da8d0927a9149e4c4145ff3 Mon Sep 17 00:00:00 2001
From: A. Gordon assafgor...@gmail.com
Date: Fri, 24 Jan 2014 13:39:14 -0500
Subject: [PATCH] scripts: add check_program, to run tests easily
* scripts/check_program: New script, so you can easily run all tests
relating to a certain program. Takes less time than checking all
programs with 'make check', and quicker to type than
'make check TESTS=TEST1,TEST2,TEST3' for multiple tests.
---
scripts/check_program | 70 +++
1 file changed, 70 insertions(+)
create mode 100755 scripts/check_program
diff --git a/scripts/check_program b/scripts/check_program
new file mode 100755
index 000..f38e410
--- /dev/null
+++ b/scripts/check_program
@@ -0,0 +1,70 @@
+#!/bin/sh
+# A small helper script to run multiple tests at once.
+# example:
+# ./scripts/check_program sort
+# would run all 'sort' related tests under ./tests/
+
+# Written by Assaf Gordon
+
+# allow the user to override 'make'
+MAKE=${MAKE-make}
+
+VERSION='2014-01-24 00:37:51' # UTC
+
+prog_name=`basename $0`
+die () { echo $prog_name: $* 2; exit 1; }
+
+usage() {
+ echo 2 \
+Usage: $0 [OPTION] PROGRAM
+Runs all tests for PROGRAM
+
+Options:
+ -e run EXPENSIVE tests
+ -v run EXPENSIVE and VERY_EXPENSIVE tests
+ -h display this help and exit
+
+Examples:
+To run all (non-expensive) tests for 'uniq':
+
+ $0 uniq
+
+To run all (including expensive and very expensive) tests for 'sort':
+
+ $0 -v sort
+
+
+}
+
+RUN_EXPENSIVE_TESTS=no
+RUN_VERY_EXPENSIVE_TESTS=no
+while getopts :evh name
+do
+case $name in
+(v)RUN_VERY_EXPENSIVE_TESTS=yes;RUN_EXPENSIVE_TESTS=yes;;
+(e)RUN_EXPENSIVE_TESTS=yes;;
+(h)usage; exit 0 ;;
+(--) shift ; break ;;
+(*)die Unknown option '$OPTARG' ;;
+esac
+shift
+done
+
+PROGRAM=$1
+[ -z $PROGRAM ] die missing PROGRAM name. See '-h' for help.
+
+
+[ -d ./tests ] || die 'tests/' directory not found. \
+Please run this script from the \
+main directory of 'Coreutils'.
+
+TESTS=$(find ./tests/ \( -name '*.sh' -o -name '*.pl' \) -print | \
+ grep -w -- $PROGRAM | paste -s -d' ')
+[ -z $TESTS ] die no tests found for '$PROGRAM'
+
+echo Running the following tests for '$PROGRAM':
+echo $TESTS | tr ' ' '\n' | sed 's/^/ /'
+
+$MAKE check TESTS=$TESTS VERBOSE=yes SUBDIRS=. \
+ RUN_EXPENSIVE_TESTS=$RUN_EXPENSIVE_TESTS \
+ RUN_VERY_EXPENSIVE_TESTS=$RUN_VERY_EXPENSIVE_TESTS
--
1.8.4.3
Re: script suggestion: 'check_program' to easily run multiple tests
On 01/24/2014 07:05 PM, Assaf Gordon wrote:
Hello,
Attached is a small script I've been using.
It helps running multiple tests for a given program.
example:
./scripts/check_program sort
Will find all sort-related tests (based on filename) and run them.
Adding -e or -v also runs expensive and very expensive tests:
examle:
./scripts/check_program -v sort
is equivalent to:
make check VERBOSE=yes SUBDIRS=. \
RUN_EXPENSIVE_TESTS=yes \ RUN_VERY_EXPENSIVE_TESTS=yes \
TESTS=./tests/misc/sort-NaN-infloop.sh
./tests/misc/sort-benchmark-random.sh
./tests/misc/sort-compress-hang.sh
./tests/misc/sort-compress-proc.sh
./tests/misc/sort-compress.sh
./tests/misc/sort-continue.sh
./tests/misc/sort-debug-keys.sh
./tests/misc/sort-debug-warn.sh
./tests/misc/sort-discrim.sh
./tests/misc/sort-exit-early.sh
./tests/misc/sort-files0-from.pl
./tests/misc/sort-float.sh
./tests/misc/sort-merge-fdlimit.sh
./tests/misc/sort-merge.pl
./tests/misc/sort-month.sh
./tests/misc/sort-rand.sh
./tests/misc/sort-spinlock-abuse.sh
./tests/misc/sort-stale-thread-mem.sh
./tests/misc/sort-u-FMR.sh
./tests/misc/sort-unique-segv.sh
./tests/misc/sort-unique.sh
./tests/misc/sort-version.sh
./tests/misc/sort.pl
If others find it useful, you're welcomed to add this.
This is a good idea.
I have something similar (attached) where I can pass a test or a test directory
or program.
The most common use I have for that is: ./make --test tests/misc/test-name.sh
Since there are multiple --test modes, I suppose we could merge both
our scripts to a ./check script?
The other modes in my `make` script would probably be best as make targets.
thanks,
Pádraig.
#!/bin/sh
# Support ./make --test tests/misc/md5sum-bsd
if [ $1 = '--test' ]; then
shift
if test -d $1; then
tests=$(shopt -s nullglob; echo tests/dd/*.{sh,pl})
elif ! printf '%s\n' $1 | grep -qF . $1; then
# Mo .suffix then look for program name
tests=$(find tests/ -name *.sh -o -name *.pl |
xargs grep -l print_ver_.*$1 | paste -s -d' ')
else
tests=$*
fi
# SUBDIRS avoids gnulib tests
make check TESTS=$tests SUBDIRS=. VERBOSE=yes RUN_VERY_EXPENSIVE_TESTS=yes
RUN_EXPENSIVE_TESTS=yes
exit
elif [ $1 = '--test-gl' ]; then
shift
gnulib/gnulib-tool --local-dir=gl --create-testdir --with-tests --test $@
elif [ $1 = '--32' ]; then
shift
# Run once without params to reset everything
# subsequent runs must both specify --32 _and_ a target
if [ $# = 0 ]; then
make clean
CFLAGS=-m32 ./configure --quiet --with-openssl=optional
fi
bitwidth=-m32
elif [ $1 = '--64' ]; then
shift
make clean
./configure --quiet --with-openssl=optional
fi
cores=$(($(nproc) * 2))
# Work around syntax-check parallel make issues
# Fixed since Sep 6 2012 with non recursive make change:
# Avoid automake update for
https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2012-3386
#[ $1 = 'syntax-check' ] sed -i 's/chmod a+w \$(distdir)/chmod u+w
\$(distdir)/' Makefile Makefile.in
# -Wno-error=suggest-attribute=pure needed for gcc = 4.6.0
_CFLAGS=$bitwidth -march=native -g -O2 -Wno-error=suggest-attribute=pure
if [ $1 = '--debug' ]; then
_CFLAGS=-ggdb
# Disable -Werror completely for debug as
# -Wuninitialized needs -O for example
WERROR='WERROR_CFLAGS='
shift
fi
make -j$cores CFLAGS=$_CFLAGS $WERROR $@
# I'm not sure why these are split out
# Maybe coreutils specific syntax checks?
[ $1 = syntax-check ] make check-local
Re: du -s excluding size from directory entries?
would people be interested in a patch implementing this? it's about a dozen lines to add (~60 line patch). On Sat, Apr 13, 2013 at 2:15 AM, Pádraig Brady p...@draigbrady.com wrote: On 04/13/2013 08:22 AM, Aaron Davies wrote: is there an easy way to `du -s' a tree and ignore the size of the directories themselves? if, e.g., i have one deep directory tree with a long history of modifications, and another directory tree created from a `cp -a' of the first, their `du -sb' results are likely to be somewhat different. i can get this result with something like `find -not -type d -print0|xargs -r0 stat -c %s|paste -sd+|bc', but a simpler solution would be nice. How about: find -not -type d -print0 | du -hc --files0-from=- | tail -n1 That deals with hard links in the set too. Thanks, Pádraig. -- Aaron Davies aaron.dav...@gmail.com
Re: script suggestion: 'check_program' to easily run multiple tests
On 01/24/2014 08:36 PM, Pádraig Brady wrote: The most common use I have for that is: ./make --test tests/misc/test-name.sh Hmm, I'd just run make check TESTS=tests/misc/test-name.sh SUBDIRS=. in that case. And when I want to run all cp tests, then I'd do something like make check TESTS=$( echo tests/cp/*.sh ) SUBDIRS=. Including expensive or very-expensive tests is not a big deal, and similar to the make target check-very-expensive. Thus said, instead of adding a script, I'd favor a new make target which can search all test cases and run them - incl. (very-)expensive if necessary. Something like make check-some T=cp Have a nice day, Berny
Re: du -s excluding size from directory entries?
On 01/24/2014 08:41 PM, Aaron Davies wrote: would people be interested in a patch implementing this? it's about a dozen lines to add (~60 line patch). On Sat, Apr 13, 2013 at 2:15 AM, Pádraig Brady p...@draigbrady.com wrote: On 04/13/2013 08:22 AM, Aaron Davies wrote: is there an easy way to `du -s' a tree and ignore the size of the directories themselves? if, e.g., i have one deep directory tree with a long history of modifications, and another directory tree created from a `cp -a' of the first, their `du -sb' results are likely to be somewhat different. i can get this result with something like `find -not -type d -print0|xargs -r0 stat -c %s|paste -sd+|bc', but a simpler solution would be nice. How about: find -not -type d -print0 | du -hc --files0-from=- | tail -n1 That deals with hard links in the set too. Does the above not suffice? If it does it's probably better than adding extra options? thanks, Pádraig.
Re: script suggestion: 'check_program' to easily run multiple tests
On 01/24/2014 10:39 PM, Bernhard Voelker wrote: On 01/24/2014 08:36 PM, Pádraig Brady wrote: The most common use I have for that is: ./make --test tests/misc/test-name.sh Hmm, I'd just run make check TESTS=tests/misc/test-name.sh SUBDIRS=. in that case. And when I want to run all cp tests, then I'd do something like make check TESTS=$( echo tests/cp/*.sh ) SUBDIRS=. Including expensive or very-expensive tests is not a big deal, and similar to the make target check-very-expensive. Thus said, instead of adding a script, I'd favor a new make target which can search all test cases and run them - incl. (very-)expensive if necessary. Something like make check-some T=cp Yes I agree. We already have some other check-targets so we should probably augment those so we have a standard interface for running tests, but also handy enough to run subsets easily. I'll see what I can come up with. thanks, Pádraig.
Re: du -s excluding size from directory entries?
On Jan 24, 2014, at 3:40 PM, Pádraig Brady p...@draigbrady.com wrote: On 01/24/2014 08:41 PM, Aaron Davies wrote: would people be interested in a patch implementing this? it's about a dozen lines to add (~60 line patch). On Sat, Apr 13, 2013 at 2:15 AM, Pádraig Brady p...@draigbrady.com wrote: On 04/13/2013 08:22 AM, Aaron Davies wrote: is there an easy way to `du -s' a tree and ignore the size of the directories themselves? if, e.g., i have one deep directory tree with a long history of modifications, and another directory tree created from a `cp -a' of the first, their `du -sb' results are likely to be somewhat different. i can get this result with something like `find -not -type d -print0|xargs -r0 stat -c %s|paste -sd+|bc', but a simpler solution would be nice. How about: find -not -type d -print0 | du -hc --files0-from=- | tail -n1 That deals with hard links in the set too. Does the above not suffice? If it does it's probably better than adding extra options? Well, it's not terrible, and it does what I asked for, but it's not exactly easy to type (or remember). With my patch, that turns into `du -sbp` . (Insert joke about GNU program options here.)
Re: du -s excluding size from directory entries?
On 01/25/2014 04:25 AM, Aaron Davies wrote: On Jan 24, 2014, at 3:40 PM, Pádraig Brady p...@draigbrady.com wrote: On 01/24/2014 08:41 PM, Aaron Davies wrote: would people be interested in a patch implementing this? it's about a dozen lines to add (~60 line patch). On Sat, Apr 13, 2013 at 2:15 AM, Pádraig Brady p...@draigbrady.com wrote: On 04/13/2013 08:22 AM, Aaron Davies wrote: is there an easy way to `du -s' a tree and ignore the size of the directories themselves? if, e.g., i have one deep directory tree with a long history of modifications, and another directory tree created from a `cp -a' of the first, their `du -sb' results are likely to be somewhat different. i can get this result with something like `find -not -type d -print0|xargs -r0 stat -c %s|paste -sd+|bc', but a simpler solution would be nice. How about: find -not -type d -print0 | du -hc --files0-from=- | tail -n1 That deals with hard links in the set too. Does the above not suffice? If it does it's probably better than adding extra options? Well, it's not terrible, and it does what I asked for, but it's not exactly easy to type (or remember). With my patch, that turns into `du -sbp` . (Insert joke about GNU program options here.) In that case I'd be leaning towards this functionality being esoteric enough that the proposed existing solution suffices. thanks, Pádraig.
bug#16532: Test Fail: tests/chown/separator
On 01/24/2014 07:43 AM, Bernhard Voelker wrote: On 01/24/2014 04:09 AM, Pádraig Brady wrote: On 01/24/2014 02:46 AM, Assaf Gordon wrote: $ getent group | grep ^users users:x:100: users:x:1000:www-data,XXX,YYY === Oh right, grep ^$id_gn: it is so. Note we can't rely on getent(1) being available (as noted in bug 9987), but the failure in that edge case is still a skip as the count will be 0 then. I'm still not convinced that we should mask this misconfiguration by skipping the test. AFAIK it's totally valid to have several names for a gid, but is it permitted the other way round? I'd bet you run into other trouble on such a system, too. Therefore, I see the failing test more as a heads-up to fix the configuration. Thanks have a nice day, Berny Since it's an easy enough workaround it's worth adding IMHO. As a general point there should never be any questions about test failures. I've seen some test suites become less than useful through attrition of little issues. For me the tests take more time than the code and are more awkward to write, but that's all fine and good as that's were the real value is. thanks, Pádraig.
bug#16532: Test Fail: tests/chown/separator
On 01/24/2014 01:19 PM, Pádraig Brady wrote: On 01/24/2014 07:43 AM, Bernhard Voelker wrote: On 01/24/2014 04:09 AM, Pádraig Brady wrote: Oh right, grep ^$id_gn: it is so. Note we can't rely on getent(1) being available (as noted in bug 9987), but the failure in that edge case is still a skip as the count will be 0 then. I'm still not convinced that we should mask this misconfiguration by skipping the test. AFAIK it's totally valid to have several names for a gid, but is it permitted the other way round? I'd bet you run into other trouble on such a system, too. Therefore, I see the failing test more as a heads-up to fix the configuration. Thanks have a nice day, Berny Since it's an easy enough workaround it's worth adding IMHO. As a general point there should never be any questions about test failures. I've seen some test suites become less than useful through attrition of little issues. For me the tests take more time than the code and are more awkward to write, but that's all fine and good as that's were the real value is. Okay, let's go with it then. Thanks. BTW: I'd change the skip_ message to reflect the above mentioned corner case where getent is not available, something like: - skip_ multiple IDs for group '$id_gn': $(getent group $id_gn) + skip_ group '$id_gn' not biunique: $(getent group | grep ^$id_gn:) Have a nice day, Berny
bug#16532: Test Fail: tests/chown/separator
On 01/24/2014 07:46 AM, Bernhard Voelker wrote: On 01/24/2014 04:09 AM, Pádraig Brady wrote: Oh right, grep ^$id_gn: it is so. BTW: I'd change the skip_ message to reflect the above mentioned corner case where getent is not available, something like: - skip_ multiple IDs for group '$id_gn': $(getent group $id_gn) + skip_ group '$id_gn' not biunique: $(getent group | grep ^$id_gn:) Attached is an updated version, using grep and the suggested error message. On addition is a tr '\n' ',' - because skip only shows the first line of the message, and so multiple groups and not displayed. -gordon From 5fc10774383928b2fbfde33de3efb314166fe475 Mon Sep 17 00:00:00 2001 From: Assaf Gordon assafgor...@gmail.com Date: Thu, 23 Jan 2014 20:34:52 -0500 Subject: [PATCH] tests: skip chown/separator with conflicting group IDs * tests/chown/separator.sh: skip test if the user's group has multiple entries. Fixes http://bugs.gnu.org/16532 --- tests/chown/separator.sh | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/chown/separator.sh b/tests/chown/separator.sh index f64b738..f5ecb5b 100755 --- a/tests/chown/separator.sh +++ b/tests/chown/separator.sh @@ -31,6 +31,12 @@ test -n $id_g || framework_failure_ id_gn=$(id -gn) || framework_failure_ test -n $id_gn || framework_failure_ +# Systems with both local and external groups with conflicting IDs, +# were seen to fail this test erroneously with EPERM errors. +test $(getent group | grep ^$id_gn: | wc -l) = 1 || + skip_ group '$id_gn' not biunique: \ +$(getent group | grep ^$id_gn: | tr '\n' ',') + # FreeBSD 6.x's getgrnam fails to look up a group name containing # a space. On such a system, skip this test if the group name contains # a byte not in the portable filename character set. -- 1.8.4.3
bug#16532: Test Fail: tests/chown/separator
On 01/24/2014 04:58 PM, Assaf Gordon wrote: On 01/24/2014 07:46 AM, Bernhard Voelker wrote: On 01/24/2014 04:09 AM, Pádraig Brady wrote: Oh right, grep ^$id_gn: it is so. BTW: I'd change the skip_ message to reflect the above mentioned corner case where getent is not available, something like: - skip_ multiple IDs for group '$id_gn': $(getent group $id_gn) + skip_ group '$id_gn' not biunique: $(getent group | grep ^$id_gn:) Attached is an updated version, using grep and the suggested error message. On addition is a tr '\n' ',' - because skip only shows the first line of the message, and so multiple groups and not displayed. If you remove the from $() the '\n' would be changed to spaces Your changing to ',' is neater though. Note to avoid a trailing ',' one can use ... | paste -s -d, Anyway pushed. thanks! Pádraig.
bug#16539: df command, possible bug?
The df command as distributed with OpenSuSE 13.1 -- package coreutils-8.21-7.8.1.x86_64 does not return all currently mounted nfs mounts when specifying the command with no command line arguments. Specifying the df command: df -at nfs works perfectly and returns the data for all mounted nfs filesystems. uname -a output for your reference: Linux otw-l0 3.11.6-4-desktop #1 SMP PREEMPT Wed Oct 30 18:04:56 UTC 2013 (e6d4a27) x86_64 x86_64 x86_64 GNU/Linux If you need further information please let me know. Thank you, Curtis -- Curtis Rubel Senior Development Engineer Compro Computer Services, Inc. 105 East Drive - Melbourne, Florida, 32904 Phone: 321-727-2211 email: cru...@compro.net Web: http://www.compro.net An ISO 9001:2008 Registered Company
bug#16539: More details on df command output for you
df command output with no args: only 1 nfs mount is listed amx@otw-l0:~ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 173232952 44850568 127310032 27% / devtmpfs 4049732 32 4049700 1% /dev tmpfs 4095336 76 4095260 1% /dev/shm tmpfs 4095336 4688 4090648 1% /run tmpfs 40953360 4095336 0% /sys/fs/cgroup tmpfs 4095336 4688 4090648 1% /var/run tmpfs 4095336 4688 4090648 1% /var/lock /dev/sda1 103508445272920848 5% /boot host:/usr/local/bin 110217536 40202928 68990448 37% /usr/local/muse amx@otw-l0:~ df command with options to show nfs mounted filesystems showing all nfs mounts. amx@otw-l0:~ df -at nfs Filesystem 1K-blocks Used Available Use% Mounted on host:/usr/local/muse110217536 40203472 68989896 37% /usr/local/muse host:/usr/local/LINUX_FILES 110217536 40203472 68989896 37% /usr/local/LINUX_FILES host:/usr/local/3rdparty110217536 40203472 68989896 37% /usr/local/3rdparty amx@otw-l0:~ Thank you. -- Curtis Rubel Senior Development Engineer Compro Computer Services, Inc. 105 East Drive - Melbourne, Florida, 32904 Phone: 321-727-2211 email: cru...@compro.net Web: http://www.compro.net An ISO 9001:2008 Registered Company --- CONFIDENTIALITY NOTICE: This email transmission, and any documents, files or previous email messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error, please immediately notify the sender by email or call 321-727-2211. Please destroy the original transmission and its attachments without reading or saving it in any manner.
bug#16539: More details on df command output for you
tag 16539 notabug close thanks On 01/24/2014 09:47 PM, Curtis Rubel wrote: df command output with no args: only 1 nfs mount is listed amx@otw-l0:~ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 173232952 44850568 127310032 27% / devtmpfs 4049732 32 4049700 1% /dev tmpfs 4095336 76 4095260 1% /dev/shm tmpfs 4095336 4688 4090648 1% /run tmpfs 40953360 4095336 0% /sys/fs/cgroup tmpfs 4095336 4688 4090648 1% /var/run tmpfs 4095336 4688 4090648 1% /var/lock /dev/sda1 103508445272920848 5% /boot host:/usr/local/bin 110217536 40202928 68990448 37% /usr/local/muse amx@otw-l0:~ df command with options to show nfs mounted filesystems showing all nfs mounts. amx@otw-l0:~ df -at nfs Filesystem 1K-blocks Used Available Use% Mounted on host:/usr/local/muse110217536 40203472 68989896 37% /usr/local/muse host:/usr/local/LINUX_FILES 110217536 40203472 68989896 37% /usr/local/LINUX_FILES host:/usr/local/3rdparty110217536 40203472 68989896 37% /usr/local/3rdparty amx@otw-l0:~ Just for info: df in openSUSE-13.1 is currently identical to that in coreutils-v8.21. The above is the result of df suppressing duplicate entries like bind mounts. This filtering is done based on the device number. As this example shows, a few exports of directories of the same file system from host are mounted - yet it's the same file system. We already had a few discussions about this filtering of duplicate mount entries. For me, this example shows that the current implementation is not that bad because df's job is to show block and inode usage statistics about mounted file systems. When it comes to information about mount points, then I think findmnt(1) from util-linux is the right tool. I'm therefore tagging this bug as notabug and mark it as done. If you or someone else wants to continue this discussion, then this can of course be done in this thread. And if someone has the *golden* idea how to solve this problem, then just stand up and let use know. Have a nice day, Berny
bug#16539: More details on df command output for you
On 01/24/2014 10:27 PM, Bernhard Voelker wrote:
tag 16539 notabug
close
thanks
On 01/24/2014 09:47 PM, Curtis Rubel wrote:
df command output with no args:
only 1 nfs mount is listed
Just for info: df in openSUSE-13.1 is currently identical to that
in coreutils-v8.21.
The above is the result of df suppressing duplicate entries like
bind mounts. This filtering is done based on the device number.
As this example shows, a few exports of directories of the same file
system from host are mounted - yet it's the same file system.
Right. Essentially df is showing storage for available file systems.
Noting that df also has a --total option, it makes sense by default
to not repeat file systems. This can be overridden easily with the
-a option as noted above.
Actually we should in fact be merging more entries!
Notice the following:
tmpfs 4095336 4688 4090648 1% /run
tmpfs 4095336 4688 4090648 1% /var/run
tmpfs 4095336 4688 4090648 1% /var/lock
Hopefully the attached patch addresses this
(and a couple of other test issues).
thanks,
Pádraig.
From 5991616451187db13ea9731ac06101081af8ccda Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com
Date: Sat, 25 Jan 2014 01:14:29 +
Subject: [PATCH] df: also deduplicate virtual file systems
* src/df.c (filter_mountlist): Remove the constraint that
a '/' needs to be in the device name for a mount entry to
be considered for deduplication. Virtual file systems also
have storage associated with them (like tmpfs for example),
and thus need to be deduplicated since they will be shown
in the default df output and subject to --total processing also.
* test/df/skip-duplicates.sh: Add a test to ensure we deduplicate
all entries, even for virtual file systems. Also avoid possible
length operations on many remote file systems in the initial
check of df operation. Also avoid the assumption that /root
is on the same file system as /.
* NEWS: Mention the change in behavior.
---
NEWS|5
src/df.c| 31 +
tests/df/skip-duplicates.sh | 45 ++
3 files changed, 51 insertions(+), 30 deletions(-)
diff --git a/NEWS b/NEWS
index 88a4154..2bf110a 100644
--- a/NEWS
+++ b/NEWS
@@ -18,6 +18,11 @@ GNU coreutils NEWS-*- outline -*-
it would display an error, requiring --no-dereference to avoid the issue.
[bug introduced in coreutils-5.3.0]
+** Changes in behavior
+
+ df now properly outputs file system information for virtual file systems
+ like tmpfs, by skipping duplicate entries (identified by the device number).
+
* Noteworthy changes in release 8.22 (2013-12-13) [stable]
diff --git a/src/df.c b/src/df.c
index e4fafb9..23b5156 100644
--- a/src/df.c
+++ b/src/df.c
@@ -630,26 +630,23 @@ filter_mount_list (void)
}
else
{
- /* If the device name is a real path name ... */
- if (strchr (me-me_devname, '/'))
+ /* If we've already seen this device... */
+ for (devlist = devlist_head; devlist; devlist = devlist-next)
+if (devlist-dev_num == buf.st_dev)
+ break;
+
+ if (devlist)
{
- /* ... try to find its device number in the devlist. */
- for (devlist = devlist_head; devlist; devlist = devlist-next)
-if (devlist-dev_num == buf.st_dev)
- break;
+ discard_me = me;
- if (devlist)
+ /* ...let the shorter mountdir win. */
+ if ((strchr (me-me_devname, '/')
+! strchr (devlist-me-me_devname, '/'))
+ || (strlen (devlist-me-me_mountdir)
+ strlen (me-me_mountdir)))
{
- discard_me = me;
-
- /* Let the shorter mountdir win. */
- if (! strchr (devlist-me-me_devname, '/')
- || (strlen (devlist-me-me_mountdir)
- strlen (me-me_mountdir)))
-{
- discard_me = devlist-me;
- devlist-me = me;
-}
+ discard_me = devlist-me;
+ devlist-me = me;
}
}
}
diff --git a/tests/df/skip-duplicates.sh b/tests/df/skip-duplicates.sh
index 266520a..b41623f 100755
--- a/tests/df/skip-duplicates.sh
+++ b/tests/df/skip-duplicates.sh
@@ -21,19 +21,26 @@
print_ver_ df
require_gcc_shared_
-df || skip_ df fails
+# We use --local here so as to not activate
+# potentially very many remote mounts.
+df --local || skip_ df fails
-# Simulate an mtab file with two entries of the same device number.
-# Also add entries with unstatable mount dirs to ensure that's handled.
+export CU_NONROOT_FS=$(df --local
