Hi Lars,
22.10.2015 17:57, Lars Ellenberg wrote:
On Wed, Oct 21, 2015 at 11:42:52AM +0300, Vladislav Bogdanov wrote:
Throughput is concurrency times request size by request completion latency.
dd has no concurrency, in the given example, request size is 1M,
if you get 450 MB/s, your latency apparently is in the order of 2.3 ms.
If you want more throughput,
you need to decrease latency, or increase concurrency.
Any suggestions on how to improve that then?
There was advise on some of related lists (may be even this) to tune
following parameters:
net/max-buffers=80k
net/max-epoch-size=20000
Tuning max-buffers is certainly needed if you have a capable backend and
replication link.
Epoch size as only a very minor effect, typical epoch sizes will be
rather in the small hundreds at most, anyways, so the
new-epoch-because-we-exceeded-the-configured-maximum will very
likely never be hit.
I can't recall details right now, but I tried several settings and this
one was chosen.
I'll do more testing (on a capable HW) in next few months.
I additionally use
disk/c-max-rate=1500m
disk/c-fill-target=0
I strongly recommend using c-fill-target, and not c-delay target,
but you may need to tune it to a value suitable for your setup.
The point was to provide/restore redundancy as fast as possible after
second node is powered on in the price of app speed.
I played several days with different settings, but this one was optimal
for my task. I got more than 1GB/s on initial sync - more than disks can
provide on writes (because of read-read verification with SSDs?).
c-max-rate is by default at 100M, so yes, if you want more than that,
you need to change it.
To clarify,
both are meaningful only in the context of a *resync*,
so are meaningless for *replication*, *cannot* have any effect
on the appliaction throughput during normal operation.
Sure.
and get >800MB/s writes with protocol C over 10Gbps Ethernet
(ConnectX-3 in ethernet or IPoIB mode) with IO depth >=32 and IO
size 512k (I use disktest from LTP project). That is what 6 Intel DC
S3610 can provide in HW RAID-10 with disk cache turned on.
And, of course, Lars is absolutely correct that it is merely
impossible to get high IO speeds with just one IO thread.
Best,
Vladislav
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
