tkhurana opened a new pull request, #2540: URL: https://github.com/apache/phoenix/pull/2540
## PHOENIX-7931: Coalesce per-batch replication appends into a single record Targets the `PHOENIX-7562-feature-new` feature branch (not master). ### What changed On the write path, a server-side batch's mutations for a given table are now coalesced into a **single** replication log record carrying a flat cell stream, rather than one record per mutation. On the replay path, `MutationCellGrouper` reconstructs Put/Delete mutations from that cell stream on the row + put-vs-delete boundary, mirroring the algorithm HBase's `ReplicationSink` uses for WALEdits. This cuts ring-buffer pressure substantially (≈5× fewer Disruptor events at high batch sizes) without changing the fsync floor — coalescing already minimized inner syncs. The on-disk record format changes accordingly (per-cell row/family/qualifier/timestamp/type/value, with a per-record table name + commitId + attributes header). ### Key files - `IndexRegionObserver` — per-(table, batch) cell aggregation before a single `append` - `MutationCellGrouper` (new) — replay-side cell-stream → mutation regrouping - `LogFileCodec` / `LogFileRecord` / `LogFile` — new coalesced record format - `ReplicationLogGroup` / `ReplicationLog` — cell-oriented append API ### Testing - New `MutationCellGrouperTest` (10 cases): empty input, single Put/Delete, contiguous same-row/type coalescing, row-change splits, Put→Delete same-row type-boundary split, three-row Put/Delete/Put, adjacent DeleteColumn/DeleteFamily subtype split, non-consecutive same-row partitioning, and cell preservation. - New `testAppendAndSyncSingleBatchRecordCount` IT pins the coalesced record-count contract with cross-cluster cell-level equality. - New `testSyncMetricsEmitted` unit test — deterministic replacement for the flaky IT metrics assertion (injects an fsync delay to clear ms-truncation; reads metrics only after a graceful close that joins the consumer). - Expanded `LogFileCodecTest` round-trip coverage for the new format (multi-family ordering, empty-record rejection, all cell types). - `testReplicationSyncPathSimulator` (opt-in) for sync-coalescing measurement under contention. ### Verification - `phoenix-core-server` compiles clean; `MutationCellGrouperTest`, `LogFileCodecTest`, and the replication unit tests pass. - `mvn spotless:check` passes on both `phoenix-core-server` and `phoenix-core`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
