On 21 mai 2014, at 14:24, Patrick Proniewski wrote:
>> - ZFS recordsize for JVM apps like ES should be default which is 4k. Also
>> with ES, important is to match ZFS recordsize with kernel page size and
>> sector size of the drive so there is no skew in the number of I/O
>> operations. Check for yourself if higher values like 8k /16k / 64k / 256k
>> gets better throughput on ES data folder. On certain striped HW RAID
>> devices it may be the case, but I doubt it (ZFS internal buffering is
>> compensating for this effect, write throughput will suffer if recordsize is
>> too high)
>
>
> My FS is (should be?) properly aligned on the physical 4K block HDD, so it
> should be quite efficient to move to a 4k blocksize ZFS volume if it's best
> for ES.
> I'll make some measurements of I/O to make sure performances are not going
> down.
> Every page size is 4k (FreeBSD 9.x):
>
> $ sysctl -a | egrep page_?size:
> vm.stats.vm.v_page_size: 4096
> hw.pagesize: 4096
> p1003_1b.pagesize: 4096
After changing recordsize to 4k and moving away/moving back my data so they are
written with the new block size, I see an impressive difference in IO and
bandwidth usage. I've tested the same kibana request (get everything, with
filter program:apache, for last 30 days)
before (recordsize=128k -> variable)
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
zdata 661G 1.17T 3 41 65.2K 267K
zdata 661G 1.17T 554 41 47.7M 351K
zdata 661G 1.17T 424 24 43.5M 725K
zdata 661G 1.17T 465 54 50.4M 838K
zdata 661G 1.17T 2 36 54.8K 179K
after (recordsize=4k -> fixed)
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
zdata 661G 1.17T 1 16 6.80K 72.4K
zdata 661G 1.17T 1.46K 15 5.88M 64.4K
zdata 661G 1.17T 3 42 12.4K 243K
Display in Kibana does not feel faster, so I guess I have another bottleneck
somewhere (network maybe, it's a remote server, over DSL). On the disk
bandwidth side, this is clearly a huge win.
Patrick
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9C24EA73-1178-44B9-AE43-A293BD3F3857%40patpro.net.
For more options, visit https://groups.google.com/d/optout.
