On March 3, 2017 9:11:30 AM EST, "Rodney W. Grimes" 
<freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
>-- Start of PGP signed section.
>[ Charset ISO-8859-1 unsupported, converting... ]
>> On 2017-Mar-02 22:19:10 -0800, "Rodney W. Grimes"
><freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
>> >> du(1) is using fts_read(3), which is based on the stat(2)
>> >> The OpenGroup defines st_blocksize as "Number of blocks allocated
>> >> this object."  In the case of ZFS, a write(2) may return before
>> >> blocks are actually allocated.  And thanks to compression, gang
>> ...
>> >My gut tells me that this is gona cause problems, is it ONLY
>> >the st_blocksize data that is incorrect then not such a big
>> >problem, or are we returning other meta data that is wrong?
>> Note that it's st_blocks, not st_blocksize.
>Yes, I just ignore that digretion, as well as the digretion into
>being anything special about this, as it just ends up calling stat(2)
>the end anyway.
>> I did an experiment, writing a (roughly) 113MB file (some data I had
>> lying around), close()ing it and then stat()ing it in a loop.  This
>> FreeBSD 10.3 with ZFS and lz4 compression.  Over the 26ms following
>> close(), st_blocks gradually rose from 24169 to 51231.  It then
>> stable until 4.968s after the close, when st_blocks again started
>> increasing until it stabilized after a total of 5.031s at 87483. 
>> on this, st_blocks reflects the actual number of blocks physically
>> written to disk.  None of the other fields in the struct stat vary.
>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>Thank you for doing the proper regression test, that satisfies me that
>we dont have a lattent bug sitting here and infact what we have is
>exposure of the kernel caching, which I might be too thrilled about,
>is just how its gona have to be.
>> The 5s delay is presumably the TXG delay (since this system is
>> unloaded).  I'm not sure why it writes roughly ? the data immediately
>> and the rest as part of the next TXG write.
>> >My expectactions of executing a stat(2) call on a file would
>> >be that the data returned is valid and stable.  I think almost
>> >any program would expect that.
>> I think a case could be made that st_blocks is a valid representation
>> of "the number of blocks allocated for this object" - with the number
>> increasing as the data is physically written to disk.  As for it
>> stable, consider a (hypothetical) filesystem that can transparently
>> migrate data between different storage media, with different
>> algorithms etc (ZFS will be able to do this once the mythical block
>> rewrite code is written).
>I could counter argue that st_blocks is:
>st_blocks   The actual number of blocks allocated for the file in
>                 512-byte units.
>Nothing in that says anything about "on disk".  So while this thing
>is sitting in memory on the TXG queue we should return the number of
>512 byte blocks used by the memory holding the data.
>I think that would be the more correct thing than exposing the
>fact this thing is setting in a write back cache to userland.

Can we compare the results of du with du -A?

Du will show compression savings, and -A wont

ZFS compresses between the write cache and the disk, so the final size may not 
be know for 5+ seconds
Allan Jude
freebsd-current@freebsd.org mailing list
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Reply via email to