[
https://issues.apache.org/jira/browse/SOLR-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14968970#comment-14968970
]
Renaud Delbru commented on SOLR-6273:
-------------------------------------
[~shalinmangar] thanks for looking into this.
Regarding performance (2 and 3), it is true that the right batch size and
scheduler delay is very important for optimal performance. With the proper
batch sizes and scheduler delays, we have seen very low update latency between
the source and target clusters. In your setup, one document was approximately
0.2kb in size, therefore the batch size was ~14kb which should correspond to
~14mb/s of transfer rate. With such a transfer rate, the replication should
have been done in a few seconds / minutes, not hours. Could you give more
information about your setup / benchmark ? Were replication turned off while
you were indexing on the source, or you turned it on after ?
In term of moving from a batch model to to a pure streaming one, this might
probably simplify the configuration on the user size, but in term of
performance, I am not sure - maybe some other people can give their opinion
here. Batch size might not use that much memory (if properly configured), and
transfer speed also (if the batch size is properly configured too). One way to
simplify also the configuration for the user is, like you proposed, having a
configurable transfer rate but with some logic to automatically adjust the
batch size and scheduler delay based on the configurable transfer rate ?
About 5, I think transfer rate is a good addition. Latency could be computed as
the QUEUES monitoring action is returning the last document timestamp.
> Cross Data Center Replication
> -----------------------------
>
> Key: SOLR-6273
> URL: https://issues.apache.org/jira/browse/SOLR-6273
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Assignee: Erick Erickson
> Attachments: SOLR-6273-trunk-testfix1.patch,
> SOLR-6273-trunk-testfix2.patch, SOLR-6273-trunk-testfix3.patch,
> SOLR-6273-trunk.patch, SOLR-6273-trunk.patch, SOLR-6273.patch,
> SOLR-6273.patch, SOLR-6273.patch, SOLR-6273.patch
>
>
> This is the master issue for Cross Data Center Replication (CDCR)
> described at a high level here:
> http://heliosearch.org/solr-cross-data-center-replication/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
