Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Carlos Maiolino]

On Tue, Oct 29, 2013 at 04:01:16PM -0600, Andreas Dilger wrote:
> 
> On Oct 29, 2013, at 11:37 AM, Jan Kara  wrote:
> 
> > On Tue 29-10-13 16:27:02, Pavel Raiskup wrote:
> >> 
> >> Well, I now recalled somehow relevant Red Hat bug, sorry I have not
> >> mentioned it before:
> >>  https://bugzilla.redhat.com/show_bug.cgi?id=757557
> >> 
> >> CC'ing fs-devel:  The question is whether that  is not a bug in
> >> filesystem — whether filesystem should not _always_ return to fstat()
> >> block count at least 1 if there are at least some data (even if these data
> >> are inlined in inode)?  Just for catching the context, this thread starts
> >> here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
> > 
> >  So 'st_blocks' should be "the number of blocks allocated to the file,
> > 512-byte units". If we are able to store the whole file within inode, we
> > have no blocks allocated and thus setting st_blocks to 0 looks as a decent
> > thing to do. Looking into filesystems where this is possible (ext4, btrfs,
> > reiserfs) they will all set st_blocks to 0 if the file body is inlined and
> > the size is smaller than 512 bytes.
> 
> One could consider that any inode with inline data still needs to have
> the inode allocated to hold the data.  
> 

> > ext4 is yet a different matter. It does really report the number of
> > allocated blocks in st_blocks so it will report 0 while data can fit into
> > the inode (whose size is configurable during fs creation, default is 256).
> > In practice that will result in reporting non-zero st_blocks for 512-byte
> > and larger files anyways so there won't be an observable difference between
> > what ext4 and btrfs / reiserfs do. But we might still want to fix up ext4
> > to be consistent with btrfs and reiserfs so that things are more
> > future-proof.
> 
> Given that tar and rsync DO exist that silently drop user data for
> files with st_blocks == 0 (we have seen this in with Lustre tests)
> it makes sense to fix the ext4_getattr() to set st_blocks = 1 for
> files with inline data.  It is already doing something similar for
> files with delalloc blocks to work around the same problem.

Blocks used to store inode metadata are not considered data blocks, so, these
are not reported. In case of delalloc, afaik, it's doing it to reserve space and
in case it's not used, well, it's removed from the counter. But, it differs from
the inline case since you'll not be using the reserved delalloc blocks (unless
the file is extended, of course).
I'm not pretty sure about this, but if you account the space used by inline data
as a 'block', you might be accounting the same block twice, one as metadata
block, and another as data block.

Also, the idea of inline data is exactly to avoid block allocation for very
small files that fit into inode's literal area, setting st_blocks to 1 would go
against this purpose. And also, free-space count might be wrong setting it to 1
too.


> 
> >> If that is not a bug in fs, is there possible to detect that particular
> >> file is completely sparse?
> >  As Joerg wrote, SEEK_DATA / SEEK_HOLE is a proper interface for this at
> > least for systems that support it. Once you have called stat(2) on the
> > file, inode will be in cache anyways so the additional cost of open(2),
> > lseek(2), close(2) won't be that big. For systems that don't support
> > SEEK_DATA / SEEK_HOLE, you can use some heuristic like:
> >  if (st.st_size < st.st_blksize || st.st_blocks > 0)
> > /* Bite the bullet and scan the data for non-zero bytes */
> >  else
> > /* Assume the file is sparse */
> 
> I don’t see how this is better than the existing heuristic:
> 
> if (st.st_blocks > 0)
> /* file has data */
> else
> /* file is sparse */
> 
> that tools/applications are already using today.  It isn’t possible
> to retroactively fix tar, rsync, etc. to use SEEK_DATA/SEEK_HOLE, but
> in the ext4 case the inline data feature is very new and it makes
> sense to fix this in the kernel.  Of course it ALSO makes sense to
> fix this in userspace to handle the (st.st_size < st.st_blksize)
> case (this is not a lot of overhead) so that the chance of losing
> data in a variety of kernel/userspace combinations is minimized.
> 
> Cheers, Andreas
> 
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Carlos

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Jan Kara]

On Tue 29-10-13 16:27:02, Pavel Raiskup wrote:
> On Tuesday, October 29, 2013 09:59:56 Pavel Raiskup wrote:
> > >  #define ST_IS_SPARSE(st)  \
> > >(ST_NBLOCKS (st)\
> > > -< ((st).st_size / ST_NBLOCKSIZE + ((st).st_size % ST_NBLOCKSIZE != 
> > > 0)))
> > > +   < ((st).st_size / ST_NBLOCKSIZE \
> > > +  + ((st).st_size % ST_NBLOCKSIZE != 0 \
> > > +  && (st).st_size / ST_NBLOCKSIZE != 0)))
> > 
> > May the st.st_size / ST_NBLOCKSIZE be greater than 1 and data still stored
> > in inode directly?  Seems like on ext4 filesystem it is not possible [1]
> > but does anybody know about exception?
> > 
> > [1] https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inline_Data
> 
> Well, I now recalled somehow relevant Red Hat bug, sorry I have not
> mentioned it before:
>   https://bugzilla.redhat.com/show_bug.cgi?id=757557
> 
> CC'ing fs-devel:  The question is whether that  is not a bug in
> filesystem — whether filesystem should not _always_ return to fstat()
> block count at least 1 if there are at least some data (even if these data
> are inlined in inode)?  Just for catching the context, this thread starts
> here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
  So 'st_blocks' should be "the number of blocks allocated to the file,
512-byte units". If we are able to store the whole file within inode, we
have no blocks allocated and thus setting st_blocks to 0 looks as a decent
thing to do. Looking into filesystems where this is possible (ext4, btrfs,
reiserfs) they will all set st_blocks to 0 if the file body is inlined and
the size is smaller than 512 bytes.

But OTOH btrfs and reiserfs *do* in fact account the amount of space
consumed by the file. It is just that stat(2) syscall reports the space in
512-byte units and for historical reasons we ended up just truncating the
byte-precision space counter instead of rounding it up (that is a mistake I
made like 10 years ago :(). It is easy enough to start reporting the value
rounded up but I'm not sure if it won't break some userspace which already
developped some dependency on this buggy kernel behavior.

ext4 is yet a different matter. It does really report the number of
allocated blocks in st_blocks so it will report 0 while data can fit into
the inode (whose size is configurable during fs creation, default is 256).
In practice that will result in reporting non-zero st_blocks for 512-byte
and larger files anyways so there won't be an observable difference between
what ext4 and btrfs / reiserfs do. But we might still want to fix up ext4
to be consistent with btrfs and reiserfs so that things are more
future-proof.

> If that is not a bug in fs, is there possible to detect that particular
> file is completely sparse?
  As Joerg wrote, SEEK_DATA / SEEK_HOLE is a proper interface for this at
least for systems that support it. Once you have called stat(2) on the
file, inode will be in cache anyways so the additional cost of open(2),
lseek(2), close(2) won't be that big. For systems that don't support
SEEK_DATA / SEEK_HOLE, you can use some heuristic like:
  if (st.st_size < st.st_blksize || st.st_blocks > 0)
/* Bite the bullet and scan the data for non-zero bytes */
  else
/* Assume the file is sparse */
Honza
-- 
Jan Kara 
SUSE Labs, CR

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Christoph Hellwig]

On Tue, Oct 29, 2013 at 04:27:02PM +0100, Pavel Raiskup wrote:
> CC'ing fs-devel:  The question is whether that  is not a bug in
> filesystem ??? whether filesystem should not _always_ return to fstat()
> block count at least 1 if there are at least some data (even if these data
> are inlined in inode)?  Just for catching the context, this thread starts
> here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
> 
> If that is not a bug in fs, is there possible to detect that particular
> file is completely sparse?

For XFS we only support inline data for non-regular files at the moment,
but for example an inode that has data stored just inline will report
zero block, and the unfinished patches to support inline regular file
data would logically do the same.

As already pointed out by Joerg we do have a proper lseek interface to
detect if there are holes.

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Andreas Dilger]

On Oct 29, 2013, at 11:37 AM, Jan Kara  wrote:

> On Tue 29-10-13 16:27:02, Pavel Raiskup wrote:
>> 
>> Well, I now recalled somehow relevant Red Hat bug, sorry I have not
>> mentioned it before:
>>  https://bugzilla.redhat.com/show_bug.cgi?id=757557
>> 
>> CC'ing fs-devel:  The question is whether that  is not a bug in
>> filesystem — whether filesystem should not _always_ return to fstat()
>> block count at least 1 if there are at least some data (even if these data
>> are inlined in inode)?  Just for catching the context, this thread starts
>> here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
> 
>  So 'st_blocks' should be "the number of blocks allocated to the file,
> 512-byte units". If we are able to store the whole file within inode, we
> have no blocks allocated and thus setting st_blocks to 0 looks as a decent
> thing to do. Looking into filesystems where this is possible (ext4, btrfs,
> reiserfs) they will all set st_blocks to 0 if the file body is inlined and
> the size is smaller than 512 bytes.

One could consider that any inode with inline data still needs to have
the inode allocated to hold the data.  

> ext4 is yet a different matter. It does really report the number of
> allocated blocks in st_blocks so it will report 0 while data can fit into
> the inode (whose size is configurable during fs creation, default is 256).
> In practice that will result in reporting non-zero st_blocks for 512-byte
> and larger files anyways so there won't be an observable difference between
> what ext4 and btrfs / reiserfs do. But we might still want to fix up ext4
> to be consistent with btrfs and reiserfs so that things are more
> future-proof.

Given that tar and rsync DO exist that silently drop user data for
files with st_blocks == 0 (we have seen this in with Lustre tests)
it makes sense to fix the ext4_getattr() to set st_blocks = 1 for
files with inline data.  It is already doing something similar for
files with delalloc blocks to work around the same problem.

>> If that is not a bug in fs, is there possible to detect that particular
>> file is completely sparse?
>  As Joerg wrote, SEEK_DATA / SEEK_HOLE is a proper interface for this at
> least for systems that support it. Once you have called stat(2) on the
> file, inode will be in cache anyways so the additional cost of open(2),
> lseek(2), close(2) won't be that big. For systems that don't support
> SEEK_DATA / SEEK_HOLE, you can use some heuristic like:
>  if (st.st_size < st.st_blksize || st.st_blocks > 0)
>   /* Bite the bullet and scan the data for non-zero bytes */
>  else
>   /* Assume the file is sparse */

I don’t see how this is better than the existing heuristic:

if (st.st_blocks > 0)
/* file has data */
else
/* file is sparse */

that tools/applications are already using today.  It isn’t possible
to retroactively fix tar, rsync, etc. to use SEEK_DATA/SEEK_HOLE, but
in the ext4 case the inline data feature is very new and it makes
sense to fix this in the kernel.  Of course it ALSO makes sense to
fix this in userspace to handle the (st.st_size < st.st_blksize)
case (this is not a lot of overhead) so that the chance of losing
data in a variety of kernel/userspace combinations is minimized.

Cheers, Andreas

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Paul Eggert]

On 10/29/2013 08:06 AM, Tim Kientzle wrote:
> $ truncate -s 10G file && echo hello >> file
>
> Are there filesystems where the 6 bytes here would be
> stored in the inode?

Not that we know of, so the current code should be OK.
Possibly such a file system could be constructed in the future,
and if so, we'll need to revisit the code.  By then, though, I
hope that tar uses (and OSes support) SEEK_HOLE and SEEK_DATA
which would make the current heuristic obsolete anyway.

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Joerg Schilling]

Pavel Raiskup  wrote:

> Well, I now recalled somehow relevant Red Hat bug, sorry I have not
> mentioned it before:
>   https://bugzilla.redhat.com/show_bug.cgi?id=757557
>
> CC'ing fs-devel:  The question is whether that  is not a bug in
> filesystem ??? whether filesystem should not _always_ return to fstat()
> block count at least 1 if there are at least some data (even if these data
> are inlined in inode)?  Just for catching the context, this thread starts
> here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html
>
> If that is not a bug in fs, is there possible to detect that particular
> file is completely sparse?

In September 2004 I defined the SEEK_HOLE/SEEK_DATA interface together with 
Jeff Bonwick. I believe that we should require that all OS that include 
filesystems that return nblocks == 0 with data in file need to implement
SEEK_HOLE/SEEK_DATA.

Jörg

-- 
 EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
   j...@cs.tu-berlin.de(uni)  
   joerg.schill...@fokus.fraunhofer.de (work) Blog: 
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Andrew J. Schorr]

On Tue, Oct 29, 2013 at 08:06:40AM -0700, Tim Kientzle wrote:
> What about this sparse file:
> 
> $ truncate -s 10G file && echo hello >> file
> 
> Are there filesystems where the 6 bytes here would be
> stored in the inode?  That would give a large sparse
> file with zero allocated blocks but not zero content.

On my netapp, that does not result in a file with zero blocks.  But I can't say
for sure it could never happen on any filesystem.  This is why I find the
current logic a bit frightening.  It is not robust to assume that a file with
zero blocks is empty.  The assertion being made is that the test in
ST_IS_SPARSE is guaranteed to protect against all cases where st_blocks is 0
but the file is not all zeroes.

>From lib/system.h:

/* Network Appliance file systems store small files directly in the
   inode if st_size <= 64; in this case the number of blocks can be
   zero.  Perhaps other file systems have similar problems; so,
   somewhat arbitrarily, do not consider a file to be sparse if
   it has no blocks but st_size < ST_NBLOCKSIZE.  */
#define ST_IS_SPARSE(st)  \
  (ST_NBLOCKS (st)\
   < ((st).st_size / ST_NBLOCKSIZE\
  + ((st).st_size % ST_NBLOCKSIZE != 0\
 && (st).st_size / ST_NBLOCKSIZE != 0)))

A problem would occur if ST_IS_SPARSE is true and ST_NBLOCKS is zero,
but the file is not completely empty.  For that to happen, I think
we need a file where ST_NBLOCKS is zero and (st_size >= ST_NBLOCKSIZE).

So the implicit assumption here is that there are no filesystems that
can store a non-empty file with length >= the block size in the inode.
I don't know if that's guaranteed always to be true.

Regards,
Andy

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Pavel Raiskup]

On Tuesday, October 29, 2013 09:59:56 Pavel Raiskup wrote:
> >  #define ST_IS_SPARSE(st)  \
> >(ST_NBLOCKS (st)\
> > -< ((st).st_size / ST_NBLOCKSIZE + ((st).st_size % ST_NBLOCKSIZE != 0)))
> > +   < ((st).st_size / ST_NBLOCKSIZE   \
> > +  + ((st).st_size % ST_NBLOCKSIZE != 0   \
> > +&& (st).st_size / ST_NBLOCKSIZE != 0)))
> 
> May the st.st_size / ST_NBLOCKSIZE be greater than 1 and data still stored
> in inode directly?  Seems like on ext4 filesystem it is not possible [1]
> but does anybody know about exception?
> 
> [1] https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inline_Data

Well, I now recalled somehow relevant Red Hat bug, sorry I have not
mentioned it before:
  https://bugzilla.redhat.com/show_bug.cgi?id=757557

CC'ing fs-devel:  The question is whether that  is not a bug in
filesystem — whether filesystem should not _always_ return to fstat()
block count at least 1 if there are at least some data (even if these data
are inlined in inode)?  Just for catching the context, this thread starts
here: http://lists.gnu.org/archive/html/bug-tar/2013-10/msg00030.html

If that is not a bug in fs, is there possible to detect that particular
file is completely sparse?

Pavel

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Tim Kientzle]

On Oct 29, 2013, at 7:10 AM, Pavel Raiskup  wrote:

>> To be clear, I can see that the fix to ST_IS_SPARSE should cause
>> dump_regular_file to be called instead of sparse_dump_file, but I still
>> wonder if it is wise to leave this logic in place.  At the very last,
>> I think a comment would be helpful to explain that this test is valid
>> only because ST_IS_SPARSE has already succeeded.
> 
> At least for the check for zero blocks in sparse file:  It is intentional
> because it makes the processing of completely sparse files to be done in
> constant time (try to archive 'file' from `truncate -s 10G file`).  This
> could be documented possibly.  Otherwise, I would not say that there is
> unclear that sparse_dump_file is supposed to be called only against real
> sparse files.

What about this sparse file:

$ truncate -s 10G file && echo hello >> file

Are there filesystems where the 6 bytes here would be
stored in the inode?  That would give a large sparse
file with zero allocated blocks but not zero content.

Tim

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Pavel Raiskup]

> To be clear, I can see that the fix to ST_IS_SPARSE should cause
> dump_regular_file to be called instead of sparse_dump_file, but I still
> wonder if it is wise to leave this logic in place.  At the very last,
> I think a comment would be helpful to explain that this test is valid
> only because ST_IS_SPARSE has already succeeded.

At least for the check for zero blocks in sparse file:  It is intentional
because it makes the processing of completely sparse files to be done in
constant time (try to archive 'file' from `truncate -s 10G file`).  This
could be documented possibly.  Otherwise, I would not say that there is
unclear that sparse_dump_file is supposed to be called only against real
sparse files.

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Andrew J. Schorr]

Hi,

On Tue, Oct 29, 2013 at 08:03:47AM -0400, Andrew J. Schorr wrote:
> Thanks for the patch, but I don't think that fixes the problem in
> sparse.c:sparse_scan_file where it says
> 
>if (ST_NBLOCKS (st->stat) == 0)
>  offset = st->stat.st_size;
>else
>  ...

To be clear, I can see that the fix to ST_IS_SPARSE should cause
dump_regular_file to be called instead of sparse_dump_file, but I still
wonder if it is wise to leave this logic in place.  At the very last,
I think a comment would be helpful to explain that this test is valid
only because ST_IS_SPARSE has already succeeded.

Regards,
Andy

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Andrew J. Schorr]

Hi Paul,

On Mon, Oct 28, 2013 at 09:19:58PM -0700, Paul Eggert wrote:
> Thanks for the bug report.  I pushed the following patch to paxutils
> and it should propagate into GNU tar in the next release.
> 
> >From 63493234ec38ad606a5f726bc82e4fe5d8661cab Mon Sep 17 00:00:00 2001
> From: Paul Eggert 
> Date: Mon, 28 Oct 2013 21:16:53 -0700
> Subject: [PATCH] paxutils: support --sparse with tiny files on Netapp filers
> 
> * lib/system.h (ST_IS_SPARSE): Port to NFS + Netapp filers,
> where a tiny file can have zero blocks but nonzero size.
> Problem reported by Andrew J. Schorr in
> .
> ---
>  lib/system.h | 9 -
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/system.h b/lib/system.h
> index ef46267..e7f531c 100644
> --- a/lib/system.h
> +++ b/lib/system.h
> @@ -389,9 +389,16 @@ extern int errno;
>  # define ST_NBLOCKSIZE 512
>  #endif
>  
> +/* Network Appliance file systems store small files directly in the
> +   inode if st_size <= 64; in this case the number of blocks can be
> +   zero.  Perhaps other file systems have similar problems; so,
> +   somewhat arbitrarily, do not consider a file to be sparse if
> +   it has no blocks but st_size < ST_NBLOCKSIZE.  */
>  #define ST_IS_SPARSE(st)  \
>(ST_NBLOCKS (st)\
> -< ((st).st_size / ST_NBLOCKSIZE + ((st).st_size % ST_NBLOCKSIZE != 0)))
> +   < ((st).st_size / ST_NBLOCKSIZE \
> +  + ((st).st_size % ST_NBLOCKSIZE != 0 \
> +  && (st).st_size / ST_NBLOCKSIZE != 0)))
>  
>  /* Declare standard functions.  */

Thanks for the patch, but I don't think that fixes the problem in
sparse.c:sparse_scan_file where it says

   if (ST_NBLOCKS (st->stat) == 0)
 offset = st->stat.st_size;
   else
 ...

Regards,
Andy

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Pavel Raiskup]

>  #define ST_IS_SPARSE(st)  \
>(ST_NBLOCKS (st)\
> -< ((st).st_size / ST_NBLOCKSIZE + ((st).st_size % ST_NBLOCKSIZE != 0)))
> +   < ((st).st_size / ST_NBLOCKSIZE \
> +  + ((st).st_size % ST_NBLOCKSIZE != 0 \
> +  && (st).st_size / ST_NBLOCKSIZE != 0)))

May the st.st_size / ST_NBLOCKSIZE be greater than 1 and data still stored
in inode directly?  Seems like on ext4 filesystem it is not possible [1]
but does anybody know about exception?

[1] https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inline_Data

Pavel

Re: [Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Paul Eggert]

Thanks for the bug report.  I pushed the following patch to paxutils
and it should propagate into GNU tar in the next release.

>From 63493234ec38ad606a5f726bc82e4fe5d8661cab Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Mon, 28 Oct 2013 21:16:53 -0700
Subject: [PATCH] paxutils: support --sparse with tiny files on Netapp filers

* lib/system.h (ST_IS_SPARSE): Port to NFS + Netapp filers,
where a tiny file can have zero blocks but nonzero size.
Problem reported by Andrew J. Schorr in
.
---
 lib/system.h | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/system.h b/lib/system.h
index ef46267..e7f531c 100644
--- a/lib/system.h
+++ b/lib/system.h
@@ -389,9 +389,16 @@ extern int errno;
 # define ST_NBLOCKSIZE 512
 #endif
 
+/* Network Appliance file systems store small files directly in the
+   inode if st_size <= 64; in this case the number of blocks can be
+   zero.  Perhaps other file systems have similar problems; so,
+   somewhat arbitrarily, do not consider a file to be sparse if
+   it has no blocks but st_size < ST_NBLOCKSIZE.  */
 #define ST_IS_SPARSE(st)  \
   (ST_NBLOCKS (st)\
-< ((st).st_size / ST_NBLOCKSIZE + ((st).st_size % ST_NBLOCKSIZE != 0)))
+   < ((st).st_size / ST_NBLOCKSIZE   \
+  + ((st).st_size % ST_NBLOCKSIZE != 0   \
+&& (st).st_size / ST_NBLOCKSIZE != 0)))
 
 /* Declare standard functions.  */
 
-- 
1.8.3.1

[Bug-tar] --sparse is broken on filesystems where small files may have zero blocks

[Andrew J. Schorr]

Hi,

I have a Netapp file server, and their filesystem is able to contain
small files of up to 64 bytes in the inode without allocating any 
blocks.

This causes --sparse to corrupt files silently due to this patch:

http://git.savannah.gnu.org/cgit/tar.git/commit/?id=a9895fd20c957ce184091672f1623a5bedd82407

Please see this bug report:

https://bugzilla.redhat.com/show_bug.cgi?id=1024095

It is unfortunately not the case that an st_blocks value of zero
means that the file is empty on all filesystems.  I don't know if there
are any other exceptions besides Netapp.

Regards,
Andy