bug#16539: More details on df command output for you
On 01/25/2014 11:55 PM, Bernhard Voelker wrote: On 01/25/2014 04:19 AM, Pádraig Brady wrote: On 01/24/2014 10:27 PM, Bernhard Voelker wrote: The above is the result of df suppressing duplicate entries like bind mounts. This filtering is done based on the device number. As this example shows, a few exports of directories of the same file system from host are mounted - yet it's the same file system. Right. Essentially df is showing storage for available file systems. Noting that df also has a --total option, it makes sense by default to not repeat file systems. This can be overridden easily with the -a option as noted above. Actually we should in fact be merging more entries! Notice the following: tmpfs 4095336 4688 4090648 1% /run tmpfs 4095336 4688 4090648 1% /var/run tmpfs 4095336 4688 4090648 1% /var/lock Hopefully the attached patch addresses this (and a couple of other test issues). Good idea. I think we're on the right way - and the patch looks good to me. Thanks for the review. I'll handle the syntax check too. However, I remember some other corner cases with eclipsed file systems in the Fedora bug tracker. I think we're quite close to solve them all this time (hopefully). The idea was to trust the order of mount entries returned by the kernel, i.e. in the loop over the mount entries, if the mount point is the same one as a previous one, then we should process the one mounted later. E.g. the situation where 2 file systems are mounted on the same mount point: $ findmnt | grep loop └─/mnt /dev/loop0 ext4 rw,relatime,data=ordered └─/mnt/dir /dev/loop1 ext4 rw,relatime,data=ordered └─/mnt/dir /dev/loop2 ext4 rw,relatime,data=ordered df - the new one with your patch - still shows the wrong device: $ src/df | grep loop /dev/loop0122835 1551112110 2% /mnt /dev/loop1122835 1550112111 2% /mnt/dir It should say /dev/loop2 here. BTW the numbers are correct. Right, that could be handled easy enough. loop1 is not accessible above and so should be hidden. But consider a bind mount resulting in something like: └─/mnt /dev/loop0 ext4 rw,relatime,data=ordered └─/mnt/dir /dev/loop1 ext4 rw,relatime,data=ordered └─/some/place/else /dev/loop1 ext4 rw,relatime,data=ordered └─/mnt/dir /dev/loop2 ext4 rw,relatime,data=ordered If we did a linear scan through that, we'd lose the /some/place/else due to it being a longer mount dir, and then also the original loop1 as we took /dev/loop2 for /mnt/dir. Seems like when discarding we would need to see if this was the last entry for a device and then see if there are any other candidate mount points for that device? thanks, Pádraig.
bug#16539: More details on df command output for you
On 01/26/2014 12:28 PM, Pádraig Brady wrote:
On 01/25/2014 11:55 PM, Bernhard Voelker wrote:
However, I remember some other corner cases with eclipsed file
systems in the Fedora bug tracker. I think we're quite close
to solve them all this time (hopefully).
The idea was to trust the order of mount entries returned by
the kernel, i.e. in the loop over the mount entries, if the
mount point is the same one as a previous one, then we should
process the one mounted later.
E.g. the situation where 2 file systems are mounted on the
same mount point:
$ findmnt | grep loop
└─/mnt /dev/loop0 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop1 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop2 ext4
rw,relatime,data=ordered
df - the new one with your patch - still shows the wrong device:
$ src/df | grep loop
/dev/loop0122835 1551112110 2% /mnt
/dev/loop1122835 1550112111 2% /mnt/dir
It should say /dev/loop2 here. BTW the numbers are correct.
BTW: the fstype is wrong, too (which can only be seen with -T or --output,
and if it differs, of course).
Right, that could be handled easy enough.
loop1 is not accessible above and so should be hidden.
But consider a bind mount resulting in something like:
└─/mnt /dev/loop0 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop1 ext4
rw,relatime,data=ordered
└─/some/place/else /dev/loop1 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop2 ext4
rw,relatime,data=ordered
If we did a linear scan through that, we'd lose the /some/place/else
due to it being a longer mount dir, and then also the original loop1
as we took /dev/loop2 for /mnt/dir.
Seems like when discarding we would need to see if this was the
last entry for a device and then see if there are any other candidate
mount points for that device?
Hi Padraig,
thanks.
Again, mount_list is a little beast - more below.
The following patch (on top of yours) would handle both cases
without a problem. Feel free to squash it in, if you like.
diff --git a/src/df.c b/src/df.c
index 23b5156..78768cc 100644
--- a/src/df.c
+++ b/src/df.c
@@ -631,9 +631,20 @@ filter_mount_list (void)
else
{
/* If we've already seen this device... */
+ struct devlist *d = NULL;
for (devlist = devlist_head; devlist; devlist = devlist-next)
if (devlist-dev_num == buf.st_dev)
- break;
+ {
+d = devlist;
+if (!STREQ (devlist-me-me_devname, me-me_devname))
+ {
+/* Fix the devname if the mount dir has been
+ mounted over by a different devname. */
+free (devlist-me-me_devname);
+devlist-me-me_devname = xstrdup (me-me_devname);
+ }
+ }
+ devlist = d;
if (devlist)
{
But there is yet another issue with the -a mode for such
over-mounted and therefore eclipsed file systems:
# Create 2 file system images: 1 ext4, 1 xfs.
$ dd if=/dev/zero bs=1M status=none count=128 of=img1
$ dd if=/dev/zero bs=1M status=none count=256 of=img2
$ mkfs -t ext4 -F img1 /dev/null 21
$ mkfs -t xfs -f img2 /dev/null 21
$ mkdir /mnt{1,2}
# Mount both on /mnt1.
$ mount -o loop img1 /mnt1
$ mount -o loop img2 /mnt1
# Mount the former (ext4) also on /mnt2 via its loop device.
$ mount /dev/loop0 /mnt2
# Result:
$ findmnt --output=TARGET,SOURCE,FSTYPE | grep loop
├─/mnt1 /dev/loop0 ext4
│ └─/mnt1/dev/loop1 xfs
└─/mnt2 /dev/loop0 ext4
Everything is fine now with the filtered df run ...
$ src/df --out -h | grep loop
/dev/loop1 xfs256K 3 256K1% 252M 13M 239M 6% -
/mnt1
/dev/loop0 ext432K11 32K1% 120M 1.6M 110M 2% -
/mnt2
...but df -a prints the wrong statistics for the over-mounted /mnt1!
$ src/df --out -h -a | grep loop
/dev/loop0 ext4 256K 3 256K1% 252M 13M 239M
6% -/mnt1
/dev/loop1 xfs 256K 3 256K1% 252M 13M 239M
6% -/mnt1
/dev/loop0 ext4 32K11 32K1% 120M 1.6M 110M
2% -/mnt2
Okay, this is nothing new.
BTW: strictly speaking, also the output of today's df -t rootfs -a
is wrong because the numbers are definitely not that of the early-boot
rootfs file system.
Now, how should df handle this?
a)
df silently filters out the mount entries of all eclipsed mount dirs,
even with -a.
-- Hmm, I think this
bug#16539: More details on df command output for you
On 01/26/2014 11:35 PM, Bernhard Voelker wrote:
On 01/26/2014 12:28 PM, Pádraig Brady wrote:
On 01/25/2014 11:55 PM, Bernhard Voelker wrote:
However, I remember some other corner cases with eclipsed file
systems in the Fedora bug tracker. I think we're quite close
to solve them all this time (hopefully).
The idea was to trust the order of mount entries returned by
the kernel, i.e. in the loop over the mount entries, if the
mount point is the same one as a previous one, then we should
process the one mounted later.
E.g. the situation where 2 file systems are mounted on the
same mount point:
$ findmnt | grep loop
└─/mnt /dev/loop0 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop1 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop2 ext4
rw,relatime,data=ordered
df - the new one with your patch - still shows the wrong device:
$ src/df | grep loop
/dev/loop0122835 1551112110 2% /mnt
/dev/loop1122835 1550112111 2% /mnt/dir
It should say /dev/loop2 here. BTW the numbers are correct.
BTW: the fstype is wrong, too (which can only be seen with -T or --output,
and if it differs, of course).
Right, that could be handled easy enough.
loop1 is not accessible above and so should be hidden.
But consider a bind mount resulting in something like:
└─/mnt /dev/loop0 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop1 ext4
rw,relatime,data=ordered
└─/some/place/else /dev/loop1 ext4
rw,relatime,data=ordered
└─/mnt/dir /dev/loop2 ext4
rw,relatime,data=ordered
If we did a linear scan through that, we'd lose the /some/place/else
due to it being a longer mount dir, and then also the original loop1
as we took /dev/loop2 for /mnt/dir.
Seems like when discarding we would need to see if this was the
last entry for a device and then see if there are any other candidate
mount points for that device?
Hi Padraig,
thanks.
Again, mount_list is a little beast - more below.
The following patch (on top of yours) would handle both cases
without a problem. Feel free to squash it in, if you like.
diff --git a/src/df.c b/src/df.c
index 23b5156..78768cc 100644
--- a/src/df.c
+++ b/src/df.c
@@ -631,9 +631,20 @@ filter_mount_list (void)
else
{
/* If we've already seen this device... */
+ struct devlist *d = NULL;
for (devlist = devlist_head; devlist; devlist = devlist-next)
if (devlist-dev_num == buf.st_dev)
- break;
+ {
+d = devlist;
+if (!STREQ (devlist-me-me_devname, me-me_devname))
+ {
+/* Fix the devname if the mount dir has been
+ mounted over by a different devname. */
+free (devlist-me-me_devname);
+devlist-me-me_devname = xstrdup (me-me_devname);
+ }
+ }
+ devlist = d;
if (devlist)
{
But there is yet another issue with the -a mode for such
over-mounted and therefore eclipsed file systems:
# Create 2 file system images: 1 ext4, 1 xfs.
$ dd if=/dev/zero bs=1M status=none count=128 of=img1
$ dd if=/dev/zero bs=1M status=none count=256 of=img2
$ mkfs -t ext4 -F img1 /dev/null 21
$ mkfs -t xfs -f img2 /dev/null 21
$ mkdir /mnt{1,2}
# Mount both on /mnt1.
$ mount -o loop img1 /mnt1
$ mount -o loop img2 /mnt1
# Mount the former (ext4) also on /mnt2 via its loop device.
$ mount /dev/loop0 /mnt2
# Result:
$ findmnt --output=TARGET,SOURCE,FSTYPE | grep loop
├─/mnt1 /dev/loop0 ext4
│ └─/mnt1/dev/loop1 xfs
└─/mnt2 /dev/loop0 ext4
Everything is fine now with the filtered df run ...
$ src/df --out -h | grep loop
/dev/loop1 xfs256K 3 256K1% 252M 13M 239M 6% -
/mnt1
/dev/loop0 ext432K11 32K1% 120M 1.6M 110M 2% -
/mnt2
...but df -a prints the wrong statistics for the over-mounted /mnt1!
$ src/df --out -h -a | grep loop
/dev/loop0 ext4 256K 3 256K1% 252M 13M
239M 6% -/mnt1
/dev/loop1 xfs 256K 3 256K1% 252M 13M
239M 6% -/mnt1
/dev/loop0 ext4 32K11 32K1% 120M 1.6M
110M 2% -/mnt2
Okay, this is nothing new.
BTW: strictly speaking, also the output of today's df -t rootfs -a
is wrong because the numbers are definitely not that of the early-boot
rootfs file system.
Now, how
bug#16561: Bug report for 'head' (and 'wc' et. al.)
forcemerge 16561 16329 stop On 01/26/2014 03:07 PM, LGUC wrote: THE INCOMPLETE ATTACMENT! (working on sunday makes not my lucky day. Sorry for the inconveniences!. Please disregard the previous 2 mails) __ Caracas, Sunday 26th, 2014 Ref: Bug report for 'head' (and 'wc' et. al.) Dear friends: Please find attached the text file 'head-tst.txt' As you easily can see, the following command fails and do not print anything, even if the file has: 6 lines, 49 words and 250 chars: 'head -n -0 head-tst.txt' The last line on the file does NOT end with a '\n', and this seems to be the base of the problem. If you add the last '\n', 'head' works pretty fine. Right that's an issue, coincidentally recently reported: http://bugs.gnu.org/16329 We'll include the fix for that soon. So this seems to be a problem with the definition of a 'text line': I guess that a line that has around 68 normal chars and 13 spaces, is a good candidate to be considered as a line. I found the same problem in several other core utils, being the most remarcable 'wc'. If you executes: 'wc head-tst.txt' you will get: 5 49 250 head-tst.txt what is wrong, as the file has six (6) lines instead of five (5). The last one line is missing due to the fact that it does not include a '\n' at the end. In 1998 I fix 'wc', and I have attached 'wc-fix.c' including only the most remarkable aspects, in case it could be of any help. So wc is different and is defined by POSIX to only count '\n' chars. So we can't change that really. We might be able to add a --visible-lines option that would handle this and also unicode line separators etc. But that would require more debate since it would be a new option. thanks, Pádraig.
