From: Yoann Congal <[email protected]>
When creating a sparse tar archive (tar -cS), the
sparse_scan_file_wholesparse() function tries to detect the case where a
sparse file has no data and is only a big hole.
This detection works by using the stat.st_blocks == 0 condition.
But this condition can be true for non empty files, for example:
* In the Linux sysfs filesystem:
$ stat /sys/kernel/btf/crc16
File: /sys/kernel/btf/crc16
Size: 23634 Blocks: 0 IO Block: 4096 regular file
^ ST_NBLOCKS() == 0
* In Linux ext4 filesystem, there is a small time window when, under
load, nblock is 0 after a file write and close. The Yocto Project CI
triggered this; for details and a reproducer, see
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16446#c2
In both cases, the ST_IS_SPARSE() check returns true.
When the detection is wrong, the tar archive contains a entirely sparse
file instead of the file data. And so, the archive is incorrect.
To fix this, remove sparse_scan_file_wholesparse() and rely on
SEEK_HOLE/SEEK_DATA for hole detection, falling back to the raw
zero-block scan where SEEK_HOLE is unavailable. The cost is that fully
sparse files on filesystems without SEEK_HOLE support are now read
instead of skipped.
Related bugs were reported 10+ years ago:
* [Bug-tar] stat() on btrfs reports the st_blocks with delay (data loss in
archivers)
https://lists.gnu.org/archive/html/bug-tar/2016-07/msg00000.html
* [Bug-tar] --sparse is broken on filesystems where small files may have zero
blocks
https://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
This thread ends with:
https://lists.gnu.org/archive/html/bug-tar/2013-10/msg00040.html:
> Not that we know of, so the current code should be OK.
> Possibly such a file system could be constructed in the future,
> and if so, we'll need to revisit the code. By then, though, I
> hope that tar uses (and OSes support) SEEK_HOLE and SEEK_DATA
> which would make the current heuristic obsolete anyway.
From man lseek:
The SEEK_HOLE and SEEK_DATA operations are supported for the
following filesystems:
o Btrfs (since Linux 3.1)
o OCFS (since Linux 3.2)
o XFS (since Linux 3.5)
o ext4 (since Linux 3.8)
o tmpfs(5) (since Linux 3.8)
o NFS (since Linux 3.18)
o FUSE (since Linux 4.5)
o GFS2 (since Linux 4.15)
This shows that there is enough SEEK_HOLE/DATA support for tar to
depend on for hole detection.
---
src/sparse.c | 24 ------------------------
1 file changed, 24 deletions(-)
diff --git a/src/sparse.c b/src/sparse.c
index 050d91da..660f5f19 100644
--- a/src/sparse.c
+++ b/src/sparse.c
@@ -266,26 +266,6 @@ sparse_scan_file_raw (struct tar_sparse_file *file)
return tar_sparse_scan (file, scan_end, NULL);
}
-static bool
-sparse_scan_file_wholesparse (struct tar_sparse_file *file)
-{
- struct tar_stat_info *st = file->stat_info;
- struct sp_array sp = {0, 0};
-
- /* Note that this function is called only for truly sparse files of size >= 1
- block size (checked via ST_IS_SPARSE before). See the thread
- http://www.mail-archive.com/[email protected]/msg04209.html for more info */
- if (ST_NBLOCKS (st->stat) == 0)
- {
- st->archive_file_size = 0;
- sp.offset = st->stat.st_size;
- sparse_add_map (st, &sp);
- return true;
- }
-
- return false;
-}
-
#ifdef SEEK_HOLE
/* Try to engage SEEK_HOLE/SEEK_DATA feature. */
static bool
@@ -346,10 +326,6 @@ sparse_scan_file_seek (struct tar_sparse_file *file)
static bool
sparse_scan_file (struct tar_sparse_file *file)
{
- /* always check for completely sparse files */
- if (sparse_scan_file_wholesparse (file))
- return true;
-
switch (hole_detection)
{
case HOLE_DETECTION_DEFAULT:
