ankitsol opened a new pull request, #7617: URL: https://github.com/apache/hbase/pull/7617
ReplicationSourceShipper currently persists replication offsets immediately after shipping WAL batches. This is unsafe for buffering ReplicationEndpoints (e.g. ContinuousBackupReplicationEndpoint) that stage WAL entries before durably flushing them. This change adds staged flush support by: - Tracking staged WAL size and last flush time in ReplicationSourceShipper - Adding optional buffer size and flush interval hooks to ReplicationEndpoint Introducing beforePersistingReplicationOffset() to allow endpoints to flush staged WAL data before offsets are persisted Default implementations preserve existing behaviour for non-buffering endpoints, which continue to persist offsets immediately. Added TestReplicationSourceShipperBufferedFlush - Verifies beforePersistingReplicationOffset() is not called for empty batches - Verifies it is called for non-empty batches when staging occurs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
