Your message dated Sun, 26 Jan 2025 22:59:58 -0500
with message-id <[email protected]>
and subject line Re: Bug#1094232: Even deleting does not produce space, but
2fsck is fine with the partition
has caused the Debian Bug report #1094232,
regarding e2fsprogs: resize2fs claims to resize partition but free space does
not change
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1094232: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1094232
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: e2fsprogs
Version: 1.47.2-1
Severity: important
I use LVM and one partition ran full. So I resized the partition,
resized the filesystem using resize2fs but the free space remains at
zero.
I did this again, (so another 20 GB), and the result is the same.
The third time I captured the output using script:
root@twentytwo:/scr# df -h media/
MDateisystem Größe Benutzt Verf. Verw% Eingehängt auf
/dev/mapper/datavg-media 2,4T 2,3T 0 100% /scr/media
root@twentytwo:/scr# umount media/
root@twentytwo:/scr# lvextend -r -L+20G -v /dev/datavg/media
Extending logical volume datavg/media to 2,38 TiB
Size of logical volume datavg/media changed from 2,36 TiB (619212 extents) to
2,38 TiB (624332 extents).
Archiving volume group "datavg" metadata (seqno 16).
Loading table for datavg-media (254:6).
Suspending datavg-media (254:6) with device flush
Resuming datavg-media (254:6).
File system ext4 found on datavg/media mounted at /scr/media.
Creating volume group backup "/etc/lvm/backup/datavg" (seqno 17).
Extending file system ext4 to 2,38 TiB (2618638204928 bytes) on
datavg/media...
Executing: /usr/libexec/lvresize_fs_helper --fsextend --fstype ext4 --lvpath
/dev/datavg/media --mountdir /scr/media
/usr/libexec/lvresize_fs_helper: execvp failed: Datei oder Verzeichnis nicht
gefunden
/usr/libexec/lvresize_fs_helper failed: 2
Failed to extend file system with lvresize_fs_helper.
File system extend error.
Logical volume datavg/media successfully resized.
root@twentytwo:/scr# resize2fs /dev/datavg/media
resize2fs 1.47.2 (1-Jan-2025)
Resizing the filesystem on /dev/datavg/media to 639315968 (4k) blocks.
The filesystem on /dev/datavg/media is now 639315968 (4k) blocks long.
root@twentytwo:/scr# mount media
root@twentytwo:/scr# df -h media/
Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
/dev/mapper/datavg-media 2,4T 2,3T 0 100% /scr/media
root@twentytwo:~# mount | grep media
/dev/mapper/datavg-media on /scr/media type ext4
(rw,nosuid,nodev,noexec,relatime,data=ordered)
I read resize2fs(8) several times, but could not find any
information on such a situation.
-- System Information:
Debian Release: trixie/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8) (ignored: LC_ALL
set to de_DE.UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages e2fsprogs depends on:
ii libblkid1 2.40.4-1
ii libc6 2.40-5
ii libcom-err2 1.47.2-1
ii libext2fs2t64 1.47.2-1
ii libss2 1.47.2-1
ii libuuid1 2.40.4-1
ii logsave 1.47.2-1
Versions of packages e2fsprogs recommends:
pn e2fsprogs-l10n <none>
Versions of packages e2fsprogs suggests:
pn e2fsck-static <none>
pn fuse2fs <none>
pn gpart <none>
ii libarchive13t64 3.7.4-1.1
ii parted 3.6-4+b1
-- no debconf information
--
Dr. Helge Kreutzmann [email protected]
Dipl.-Phys. http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
Help keep free software "libre": http://www.ffii.de/
signature.asc
Description: PGP signature
--- End Message ---
--- Begin Message ---
On Sun, Jan 26, 2025 at 11:49:22AM +0000, Helge Kreutzmann wrote:
> 607614353 blocks used (95.04%, out of 639315968)
There's your problem. The file system is over 95% full. By default,
the last 5% of the file system is reserved for root processes. There
are two reasons for this. The first is to make sure that the system
can function correctly by being able to write to log files and other
critical system state files when the root file system gets full. The
other reason is because the file system performance starts to suffer
when the file system gets close to 100% full, due to the file system
getting more and more fragmented when it runs close to 100% full.
In fact, on BSD Unix, the default was to use a reserve of 20%. This
is because even running at 90 to 95% full for long periods of time for
many workloads will lead to file system fragmentation, especially on
time-sharing systems, which is what was commonly used in the mid
1980's during the era of BSD 4.3. By the time we were developing the
ext2 and ext3 file systems in the 1990's, disks had gotten a lot
larger, and users would tend to get.... cranky.... if the user created
a file system on 1TB, and they found out that 200GB of the space was
not available for the user to use. Furthermore, time-sharing systems
were out, and single-user systems were much more of the norm. And it
was actually pretty common that people would by a huge disk, say, 1TB
or 4TB (reember this is in the mid 1990's), and in practice, they
wouldn't actually use more than 50% or 60% of the space, since disk
sizes were doubling every 12 to 18 months, and the days of ripping
DVD's hadn't arrived yet. :-)
These days, the problem that we have are users are using LVM, or cloud
based disks, and for some reason, they really like to grow the file
system in very tiny increments --- in the most pathological cases, in
10MB increments, but even on a 1TB file system, if you only grow it by
20GB, that's only 2% or so at a time. This turns out to very often
lead to pathological file space fragmentation for many workloads, and
then performance *really* goes down the tubes.
Ironically, HDD space is **cheap** so by trying to save a few pennies
by growing the file system by tiny amounts, when performance becomes
terrible, users will then spend $$$ on SSD's, or SSD caches, when
perhaps if they hadn't been quite so parsimonious about doling out
extra space to the file system by tiny drips and drabs, perhaps
performance would have been adequate.
In any case, what you are getting confused by is the fact that the
statfs(2) system call, which is used by df, will subtract out the
reserve space when reporting free space. This behavior comes from BSD
Unix, and by default we follow this behavior, which is what most Unix
systems do. However, if you don't like this, you can use the
"minixdf" mount option. From the ext4 man page:
bsddf|minixdf
Set the behavior for the statfs system call. The minixdf
behavior is to return in the f_blocks field the total
number of blocks of the file system, while the bsddf
behavior (which is the default) is to subtract the
overhead blocks used by the ext2 file system and not
available for file storage. Thus
% mount /k -o minixdf; df /k; umount /k
File System 1024-blocks Used Available Capacity Mounted on
/dev/sda6 2630655 86954 2412169 3% /k
% mount /k -o bsddf; df /k; umount /k
File System 1024-blocks Used Available Capacity Mounted on
/dev/sda6 2543714 13 2412169 0% /k
Note that the minixdf mount option only controls what df reports. It
doesn't change the fact that if the free space is less than the
reserved space, non-root process will get an ENOSPC when they try to
allocate blocks in the file system. So this will lead to users who
don't understand the reserved space and the reasons to complain and
file indignant bug reports when df shows that there is "free space",
but they still can't write to the file system. It is for this reason
that 40 years ago, BSD Unix subtracted out the reserved space, and why
Linux uses "bsddf" as the default.
Now, you can change the reserved space percentage using the -m option
to mke2fs or tune2fs. So if you really don't like this behaviour, you
*can* set the reserve space percentage to zero. But note that this
will risk very free space fragmentation leading to potentially
significant performance drops.
It might not matter if file system is being used to store archival
media files. In that case, setting the reserve space to zero might
make sense. However, if you are regularly creating and deleting files
of varying sizes, after enough time, I've seen performance drops
starting around 70% as the file system ages. The question of how
quickly performance drops, and by how much, very much depends on the
workload. So the BSD Unix default of using the reserve space of 80%
wasn't insane. People of good will can disagree about what the
appropriate reserve space percentage should be. Arguably, a default
of 20% or 5% is somewhat arbitrary.
But I can definitely say that if you increase the file system by 1%
and 2% full, and then fill that space, and then add another 1% or 2%,
and repeat for a long time, the files that you write, and the free
space, *will* get fragmented, and performance *will* go down over
time. Of course, perhaps saving a few pennies is more important to
you than performance --- in which case, go right ahead and do that. :-)
Anyway, this isn't a bug; just a misunderstsanding of how Linux and
Unix systems report free space and how reserved free space works.
Cheers,
- Ted
--- End Message ---
