Rahul Kumar created PHOENIX-8060:
------------------------------------
Summary: Log row-level mismatches in PhoenixSyncTableTool
Key: PHOENIX-8060
URL: https://issues.apache.org/jira/browse/PHOENIX-8060
Project: Phoenix
Issue Type: Improvement
Affects Versions: 5.2.3, 5.4.0, 5.3.3
Reporter: Rahul Kumar
Assignee: Rahul Kumar
Fix For: 5.2.3, 5.4.0, 5.3.3
When {{PhoenixSyncTableTool}} finds a chunk hash mismatch, it only logs the
chunk boundaries. Operators cannot tell which rows drifted, or whether the
drift is a missing row, an extra row, or a cell difference.
Change
On a mismatched chunk, scan the source and target row by row and classify each
row:
* Missing on target — present on source, absent on target
* Extra on target — present on target, absent on source
* Different — present on both, cells do not match
Log the row key for each drifted row and a per-chunk summary. Cell values are
not logged, and the path issues no mutations. The target scan uses the same
widened boundaries, time range, raw-scan, versions, and Phoenix TTL settings as
the hash verification, so the diff explains the mismatch the verifier saw.
A failure in this diagnostic scan is logged and swallowed so it does not fail
the chunk. Counts collected before the failure are still kept.
Counters
Add MapReduce counters {{{}ROWS_MISSING_ON_TARGET{}}},
{{{}ROWS_EXTRA_ON_TARGET{}}}, and {{{}ROWS_DIFFERENT{}}}. Persist the same
counts on chunk and mapper checkpoint rows, and include them in the
job-completion log. Verified chunks store zeros for these fields.
Also pulse the task during the target region scan and the row-diff walk so a
slow scan is not killed by the task timeout.
Tests
Existing sync ITs assert the new counters for modified, deleted, extra, and
timestamp-only differences. {{testSyncTableValidateWithMixedRowDiffs}} covers
one missing row, two extra rows, and one modified row in a single run.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)