Re: Is autodefrag recommended? -- re-duplication???

[Hugo Mills]

On Tue, Sep 05, 2017 at 05:01:10PM +0300, Marat Khalili wrote:
> Dear experts,
> 
> At first reaction to just switching autodefrag on was positive, but
> mentions of re-duplication are very scary. Main use of BTRFS here is
> backup snapshots, so re-duplication would be disastrous.
> 
> In order to stick to concrete example, let there be two files, 4KB
> and 4GB in size, referenced in read-only snapshots 100 times each,
> and some 4KB of both files are rewritten each night and then another
> snapshot is created (let's ignore snapshots deletion here). AFAIU
> 8KB of additional space (+metadata) will be allocated each night
> without autodefrag. With autodefrag will it be perhaps 4KB+128KB or
> something much worse?

   I'm going for 132 KiB (4+128).

   Of course, if there's two 4 KiB writes close together, then there's
less overhead, as they'll share the range.

   Hugo.

-- 
Hugo Mills | Once is happenstance; twice is coincidence; three
hugo@... carfax.org.uk | times is enemy action.
http://carfax.org.uk/  |
PGP: E2AB1DE4  |


signature.asc
Description: Digital signature

Re: Is autodefrag recommended? -- re-duplication???

[Marat Khalili]

Dear experts,

At first reaction to just switching autodefrag on was positive, but 
mentions of re-duplication are very scary. Main use of BTRFS here is 
backup snapshots, so re-duplication would be disastrous.


In order to stick to concrete example, let there be two files, 4KB and 
4GB in size, referenced in read-only snapshots 100 times each, and some 
4KB of both files are rewritten each night and then another snapshot is 
created (let's ignore snapshots deletion here). AFAIU 8KB of additional 
space (+metadata) will be allocated each night without autodefrag. With 
autodefrag will it be perhaps 4KB+128KB or something much worse?


--

With Best Regards,
Marat Khalili

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Austin S. Hemmelgarn]

On 2017-09-05 08:49, Henk Slager wrote:

On Tue, Sep 5, 2017 at 1:45 PM, Austin S. Hemmelgarn
 wrote:


   - You end up duplicating more data than is strictly necessary. This
 is, IIRC, something like 128 KiB for a write.


FWIW< I'm pretty sure you can mitigate this first issue by running a regular
defrag on a semi-regular basis (monthly is what I would probably suggest).


No, both autodefrag and regular defrag duplicate data, so if you keep
snapshots around for weeks or months, it can eat up a significant
amount of space.

I'm not talking about data duplication due to broken reflinks, I'm 
talking about data duplication due to how partial extent rewrites are 
handled in BTRFS.


As a more illustrative example, suppose you've got a 256k file that has 
just one extent.  Such a file will require 256k of space for the data 
Now rewrite from 128k to 192k.  The file now technically takes up 320k, 
because the region you rewrote is still allocated in the original extent.


I know that sub-extent-size reflinks are handled like this (in the above 
example, if you instead use the CLONE ioctl to create a new file 
reflinking that range, then delete the original, the remaining 192k of 
space in the extent ends up unreferenced, but gets kept around until the 
referenced region is no longer referenced (and the easiest way to ensure 
this is to either rewrite the whole file, or defragment it)), and I'm 
pretty sure from reading the code that mid-extent writes are handled 
this way too, in which case, a full defrag can reclaim that space.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Henk Slager]

On Tue, Sep 5, 2017 at 1:45 PM, Austin S. Hemmelgarn
 wrote:

>>   - You end up duplicating more data than is strictly necessary. This
>> is, IIRC, something like 128 KiB for a write.
>
> FWIW< I'm pretty sure you can mitigate this first issue by running a regular
> defrag on a semi-regular basis (monthly is what I would probably suggest).

No, both autodefrag and regular defrag duplicate data, so if you keep
snapshots around for weeks or months, it can eat up a significant
amount of space.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[A L]

There is a drawback in that defragmentation re-dups data that is previously 
deduped or shared in snapshots/subvolumes.

 From: Marat Khalili  -- Sent: 2017-09-04 - 11:31 

> Hello list,
> good time of the day,
> 
> More than once I see mentioned in this list that autodefrag option 
> solves problems with no apparent drawbacks, but it's not the default. 
> Can you recommend to just switch it on indiscriminately on all 
> installations?
> 
> I'm currently on kernel 4.4, can switch to 4.10 if necessary (it's 
> Ubuntu that gives us this strange choice, no idea why it's not 4.9). 
> Only spinning rust here, no SSDs.
> 
> --
> 
> With Best Regards,
> Marat Khalili
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Austin S. Hemmelgarn]

On 2017-09-04 06:54, Hugo Mills wrote:

On Mon, Sep 04, 2017 at 12:31:54PM +0300, Marat Khalili wrote:

Hello list,
good time of the day,

More than once I see mentioned in this list that autodefrag option
solves problems with no apparent drawbacks, but it's not the
default. Can you recommend to just switch it on indiscriminately on
all installations?

I'm currently on kernel 4.4, can switch to 4.10 if necessary (it's
Ubuntu that gives us this strange choice, no idea why it's not 4.9).
Only spinning rust here, no SSDs.


autodefrag effectively works by taking a small region around every
write or cluster of writes and making that into a stand-alone extent.
I was under the impression that it had some kind of 'random access' 
detection heuristic, and onky triggered if that flagged the write 
patterns as 'random'.


This has two consequences:

  - You end up duplicating more data than is strictly necessary. This
is, IIRC, something like 128 KiB for a write.
FWIW< I'm pretty sure you can mitigate this first issue by running a 
regular defrag on a semi-regular basis (monthly is what I would probably 
suggest).


  - There's an I/O overhead for enabling autodefrag, because it's
increasing the amount of data written.
And this issue may not be as much of an issue.  The region being 
rewritten gets written out sequentially, so it will increase the amount 
of data written, but in most cases probably won't increase IO request 
counts to the device by much.  If you care mostly about raw bandwidth, 
then this could still have an impact, but if you care about IOPS, it 
probably won't have much impact unless you're already running the device 
at peak capacity.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Duncan]

Henk Slager posted on Mon, 04 Sep 2017 13:09:24 +0200 as excerpted:

> On Mon, Sep 4, 2017 at 12:34 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> 
>> * Autodefrag works very well when these internal-rewrite-pattern files
>> are relatively small, say a quarter GiB or less, but, again with near-
>> capacity throughput, not necessarily so well with larger databases or
>> VM images of a GiB or larger.  (The quarter-gig to gig size is
>> intermediate,
>> not as often a problem and not a problem for many, but it can be for
>> slower devices, while those on fast ssds may not see a problem until
>> sizes reach multiple GiB.)
> 
> I have seen you stating this before about some quarter GiB filesize or
> so, but it is irrelevant, it is simply not how it works. See explanation
> of Hugo for how it works. I can post/store an actual filefrag output of
> a vm image that is around for 2 years on the one of my btrfs fs, then
> you can do some statistics on it and see from there how it works.

FWIW...

I believe it did work that way (whole-file autodefrag) at one point.  
Because back in the early kernel 3.x era at least, we had complaints 
about autodefrag performance with larger internal-rewrite-pattern files 
where the larger the file the worse the performance, and the 
documentation mentioned something about being appropriate for small files 
but less so far large files as well.

But I also believe you're correct that it no longer works that way (if it 
ever did, maybe the complaints were due to some unrelated side effect, in 
any case I've not seen any for quite some time now), and hasn't since 
before anything we're still trying to reasonably support on this list 
(IOW, back two LTS kernel series ago, so to 4.4).

So I should drop the size factor, or mention that it's not nearly the 
problem it once was, at least.

Thanks for forcing the reckoning. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Henk Slager]

On Mon, Sep 4, 2017 at 12:34 PM, Duncan <1i5t5.dun...@cox.net> wrote:

> * Autodefrag works very well when these internal-rewrite-pattern files
> are relatively small, say a quarter GiB or less, but, again with near-
> capacity throughput, not necessarily so well with larger databases or VM
> images of a GiB or larger.  (The quarter-gig to gig size is intermediate,
> not as often a problem and not a problem for many, but it can be for
> slower devices, while those on fast ssds may not see a problem until
> sizes reach multiple GiB.)

I have seen you stating this before about some quarter GiB filesize or
so, but it is irrelevant, it is simply not how it works. See
explanation of Hugo for how it works. I can post/store an actual
filefrag output of a vm image that is around for 2 years on the one of
my btrfs fs, then you can do some statistics on it and see from there
how it works.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is autodefrag recommended?

[Hugo Mills]

On Mon, Sep 04, 2017 at 12:31:54PM +0300, Marat Khalili wrote:
> Hello list,
> good time of the day,
> 
> More than once I see mentioned in this list that autodefrag option
> solves problems with no apparent drawbacks, but it's not the
> default. Can you recommend to just switch it on indiscriminately on
> all installations?
> 
> I'm currently on kernel 4.4, can switch to 4.10 if necessary (it's
> Ubuntu that gives us this strange choice, no idea why it's not 4.9).
> Only spinning rust here, no SSDs.

   autodefrag effectively works by taking a small region around every
write or cluster of writes and making that into a stand-alone extent.

   This has two consequences:

 - You end up duplicating more data than is strictly necessary. This
   is, IIRC, something like 128 KiB for a write.

 - There's an I/O overhead for enabling autodefrag, because it's
   increasing the amount of data written.

   Hugo.

-- 
Hugo Mills | The future isn't what it used to be.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4  |


signature.asc
Description: Digital signature

Re: Is autodefrag recommended?

[Henk Slager]

On Mon, Sep 4, 2017 at 11:31 AM, Marat Khalili  wrote:
> Hello list,
> good time of the day,
>
> More than once I see mentioned in this list that autodefrag option solves
> problems with no apparent drawbacks, but it's not the default. Can you
> recommend to just switch it on indiscriminately on all installations?

Of course it has drawbacks, it depends on the use-cases on the
filesystem what your trade-off is. If the filesystem is created log
time ago and has 4k leafes the on HDD over time you get exessive
fragmentation and scattered 4k blocks all over the disk for a file
with a lot random writes (standard CoW for whole fs), like a 50G vm
image, easily 500k extents.

With autodefrag on from the beginning of fs creation, most
extent/blocksizes will be 128k or 256k in that order and then amount
of extents for the same vm image is roughly 50k. So statistically, the
average blocksize is not 4k but 128k, which is at least less free
space fragmentation (I use SSD caching of HDD otherwise also those 50k
extents result in totally unacceptable performance). But also for
newer standard 16k leafes, it is more or less the same story.

The drawbacks for me are:
1. I use nightly differential snapshotting for backup/replication over
a metered mobile network link, and this autodefrag causes a certain
amount of unnessccesaty fake content difference due to send|receive
based on CoW. But the amount of extra datavolume it causes is still
acceptable. If I would let the guest OS defragment its fs inside the
vm image for example, then the datavolume per day becomes
unacceptable.
2. It causes extra HDD activity, so noise, powerconsumption etc, which
might be unacceptable for some use-cases.

> I'm currently on kernel 4.4, can switch to 4.10 if necessary (it's Ubuntu
> that gives us this strange choice, no idea why it's not 4.9). Only spinning
> rust here, no SSDs.
Kernel 4.4 is new enough w.r.t. autodefrag, but if you can switch to
4.8 or newer, I would do so.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Is autodefrag recommended?

[Marat Khalili]

Hello list,
good time of the day,

More than once I see mentioned in this list that autodefrag option 
solves problems with no apparent drawbacks, but it's not the default. 
Can you recommend to just switch it on indiscriminately on all 
installations?


I'm currently on kernel 4.4, can switch to 4.10 if necessary (it's 
Ubuntu that gives us this strange choice, no idea why it's not 4.9). 
Only spinning rust here, no SSDs.


--

With Best Regards,
Marat Khalili
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html