sryanyuan opened a new pull request, #3340: URL: https://github.com/apache/kvrocks/pull/3340
In Kvrocks replication, a replica's requested incremental sequence must align with the start sequence of a WriteBatch in the master's WAL. For example, if a WriteBatch starts at sequence 10 and contains 3 records, valid starting points are 10 or 13. Requests starting at 11 or 12 are considered invalid, causing incremental sync to fail and triggering a costly full sync. A common cause of misaligned sequences occurs during master-slave failover: if the old master still has WAL entries that were not yet replicated to the old slave when the role switch happens, and the new master continues to accept writes, then re-establishing replication from this new master can result in the replica requesting a sequence that falls inside an existing WriteBatch. This mismatch forces a full sync, even if only a few records are missing. This change introduces an optional configuration `replication-enable-sequence-padding` (default: no). When enabled, the master will send dummy WriteBatch entries to pad the replication stream, advancing the replica's sequence to the next valid position. This allows incremental sync to continue while skipping the missing records, avoiding full sync when only a small number of logs are lost. Trade-off: skipped records are not applied on the replica, potentially causing minor data inconsistency. This configuration is mainly intended for cache-like use cases, where the application can tolerate partial data loss or temporary inconsistency in favor of maintaining replication continuity and avoiding expensive full syncs. Changes: - Added `replication-enable-sequence-padding` config and documentation. - Implemented padding logic in replication send path. - Added unit tests `TestReplicationSequencePadding` to verify behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
