On 15.02.21 г. 16:53 ч., Pal, Laszlo wrote:
> Hi,
>
> I'm not sure this is the right place to ask, but let me try :) I have
> a server where I mainly using btrfs because of the builtin compress
> feature. This is a central log server, storing logs from tens of
> thousands devices, using a text files in thousands of directories in
> millions of files.
>
> I've started to think it was not the best idea to choose btrfs for this :)
>
> The performance of this server was always worst than others where I
> don't use btrfs, but I thought this is just because the i/o overhead
> of compression and the not-so-good esx host providing the disk to this
> machine. But now, even rm a single file takes ages, so there is
> something definitely wrong. So, I'm looking for some recommendations
> for such an environment where the data-security functions of btrfs is
> not as important than the performance.
>
> I was searching the net for some comprehensive performance documents
> for months, but I cannot find it so far.
>
> Thank you in advance
> Laszlo
>
You are likely suffering fragmentation issues, given you hold log files
I'd assume you do a lot of small writes, each one results in a CoW
operation which allocates space. This results in increasing the size of
the metadata tree and since you are likely using harddrives seeking is
slow. To try and ascertain if that's really the case I'd advise you to
show the output of the following commands:
btrfs fi usage <mountpoint> - this will show the total used space on the
filesystem.
Then run btrfs inspect-internal dump-tree -t5 </dev/xxx> | grep -c
EXTENT_DATA
Which will show how many data extents there are in the filesystem.
Subsequently run btrfs inspect-internal dump-tree -t5 </dev/xxx> | grep
-c leaf which will show how many leaves there are in the filesystem.
Then you have 2 options:
a) Use btrfs defragment to actually rewrite leaves to try and make them
be closer so that seeks are going to become somewhat cheaper,
b) Rewrite the logs files by copying them with no reflinks so that
instead of 1 file consisting of multiple small extents just make them
consist of 1 giant extent, also with your use case I'd assume you also
want nocow to be enabled, unfortunately nodatacow precludes using
compression.
