ankitsol commented on code in PR #7617:
URL: https://github.com/apache/hbase/pull/7617#discussion_r2736351890
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java:
##########
@@ -229,6 +247,41 @@ private void shipEdits(WALEntryBatch entryBatch) {
}
}
+ private boolean shouldPersistLogPosition() {
+ ReplicationEndpoint endpoint = source.getReplicationEndpoint();
Review Comment:
Ok, so we need to have these conf (time and size ones) at both
(ReplicationSourceShipper and ReplicationEndpoint) ends, so we can achieve the
tuning you mention in your previous comment:
https://github.com/apache/hbase/pull/7617#discussion_r2719322657
`The design here is that, when using different ReplicationEndpoint, you need
to tune the shipper configuration by your own, as the parameters are not only
affected by ReplicationEndpoint, they also depend on the shipper side.
For example, when you want to reduce the pressure on recording the offset,
you should increase the record interval, i.e, increase batch size, increase the
number of ship times between recording offset, etc. And if you want to reduce
the pressure on memory and the target receiver, you should decrease the batch
size, and for S3 based replication endpoint, there is also a trade off, if you
increase the flush interval, you can get better performance and less files on
S3, but failover will be more complicated as you need to start from long
before.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
