Rahul Kumar created PHOENIX-8064:
------------------------------------
Summary: Sample PhoenixSyncTableTool row-diff logs
Key: PHOENIX-8064
URL: https://issues.apache.org/jira/browse/PHOENIX-8064
Project: Phoenix
Issue Type: Improvement
Affects Versions: 5.2.3, 5.4.0, 5.3.3
Reporter: Rahul Kumar
Assignee: Rahul Kumar
Fix For: 5.2.3, 5.4.0, 5.3.3
On a chunk mismatch, {{PhoenixSyncTableTool}} logs every drifted row key. A
large drift writes one warn line per row and floods the mapper logs. The
counters and the per-chunk summary already give the full counts.
Change
Add {{--row-diff-log-sample-rate}}
({{{}phoenix.sync.table.row.diff.log.sample.rate{}}}). Each mapper always logs
the first mismatch, then every Nth. The default is 1000. A value of 1 logs
every row. Zero and negative values are rejected.
MapReduce counters and the per-chunk summary stay unsampled. Only the per-row
warn line is sampled. The log line includes the mismatch number so the sample
can be tied back to the full count.
Tests
Unit tests cover the first-mismatch rule, the default rate of 1000, rate 1,
values between sample points, CLI parsing, rejection of 0 and negative rates,
and the configuration default.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)