For better or worse, out of the box, librbd and rbd-mirror are configured to conserve memory at the expense of performance to support the potential case of thousands of images being mirrored and only a single "rbd-mirror" daemon attempting to handle the load.
You can optimize writes by adding "rbd_journal_max_payload_bytes = 8388608" to the "[client]" section on the librbd client nodes. Normally, writes larger than 16KiB are broken into multiple journal entries to allow the remote "rbd-mirror" daemon to make forward progress w/o using too much memory, so this will ensure large IOs only require a single journal entry. You can also add "rbd_mirror_journal_max_fetch_bytes = 33554432" to the "[client]" section on the "rbd-mirror" daemon nodes and restart the daemon for the change to take effect. Normally, the daemon tries to nibble the per-image journal events to prevent excessive memory use in the case where potentially thousands of images are being mirrored. On Wed, Apr 3, 2019 at 4:34 PM huxia...@horebdata.cn <huxia...@horebdata.cn> wrote: > > Hello, folks, > > I am setting up two ceph clusters to test async replication via RBD > mirroring. The two clusters are very close, just in two buildings about 20m > away, and the networking is very good as well, 10Gb Fiber connection. In this > case, how should i tune the relevant RBD mirroring parameters to accelerate > the replication? > > thanks in advance, > > Samuel > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Jason _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
