Rahul Kumar created PHOENIX-8062:
------------------------------------
Summary: PhoenixSyncTableTool rescans rows after a paging dummy row
Key: PHOENIX-8062
URL: https://issues.apache.org/jira/browse/PHOENIX-8062
Project: Phoenix
Issue Type: Improvement
Affects Versions: 5.2.3, 5.4.0, 5.3.3
Reporter: Rahul Kumar
Assignee: Rahul Kumar
Fix For: 5.2.3, 5.4.0, 5.3.3
When Phoenix server paging hits its page timeout while walking filtered rows
(for example a run of tombstones), the lower scanner returns a dummy row.
{{PhoenixSyncTableRegionScanner}} stopped the chunk there and sent the metadata
back with the result row key set to the last real hashed row.
{{ClientScanner}} resumes the next RPC from that result row key. The server
therefore re-walked every row it had already hashed, hit the same dummy row
again, and could loop instead of moving past it. Rows after the dummy were
delayed or never reached, and the sync job made no forward progress on that
region.
Change
Track {{lastScannedKey}} separately from the hashed {{{}endKey{}}}.
* A real row advances both.
* A dummy row advances only {{lastScannedKey}} to the dummy key. {{endKey}}
stays on the last hashed row, and the digest is not updated.
* If the chunk has no real rows yet, emit a zero-row progress marker at the
dummy key.
The chunk metadata result row key is now {{{}lastScannedKey{}}}, so the client
resumes past the dummy. The real hashed boundary is stored in a new {{END_KEY}}
cell. The mapper reads {{endKey}} from that cell and {{lastScannedKey}} from
the result row, and the next target scan starts exclusive of
{{{}lastScannedKey{}}}. Zero-row markers are skipped; they only move the client
scanner forward.
Cell tags are included in the chunk hash, so a tag-only difference is reported
as a mismatch.
The job completion log also reports mappers entered and mappers completed.
Tests
* Unit tests for {{lastScannedKey}} versus {{{}endKey{}}}, the dummy progress
marker, tag bytes in the digest, and the mapper skipping zero-row results.
* {{{}testSyncTableValidateWithOnlyCellTagDifferences{}}}: identical cells
that differ only by a tag mismatch on every row.
* {{{}testSyncTableValidatePagingDummyRowSurfacedChunkSize10MB{}}}: dense
tombstones with server paging enabled. The job finishes, counts only the live
rows, and reports no mismatched chunks.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)