Rahul Kumar created PHOENIX-8062:
------------------------------------

             Summary: PhoenixSyncTableTool rescans rows after a paging dummy row
                 Key: PHOENIX-8062
                 URL: https://issues.apache.org/jira/browse/PHOENIX-8062
             Project: Phoenix
          Issue Type: Improvement
    Affects Versions: 5.2.3, 5.4.0, 5.3.3
            Reporter: Rahul Kumar
            Assignee: Rahul Kumar
             Fix For: 5.2.3, 5.4.0, 5.3.3


When Phoenix server paging hits its page timeout while walking filtered rows 
(for example a run of tombstones), the lower scanner returns a dummy row. 
{{PhoenixSyncTableRegionScanner}} stopped the chunk there and sent the metadata 
back with the result row key set to the last real hashed row.

{{ClientScanner}} resumes the next RPC from that result row key. The server 
therefore re-walked every row it had already hashed, hit the same dummy row 
again, and could loop instead of moving past it. Rows after the dummy were 
delayed or never reached, and the sync job made no forward progress on that 
region.

Change

Track {{lastScannedKey}} separately from the hashed {{{}endKey{}}}.
 * A real row advances both.
 * A dummy row advances only {{lastScannedKey}} to the dummy key. {{endKey}} 
stays on the last hashed row, and the digest is not updated.
 * If the chunk has no real rows yet, emit a zero-row progress marker at the 
dummy key.

The chunk metadata result row key is now {{{}lastScannedKey{}}}, so the client 
resumes past the dummy. The real hashed boundary is stored in a new {{END_KEY}} 
cell. The mapper reads {{endKey}} from that cell and {{lastScannedKey}} from 
the result row, and the next target scan starts exclusive of 
{{{}lastScannedKey{}}}. Zero-row markers are skipped; they only move the client 
scanner forward.

Cell tags are included in the chunk hash, so a tag-only difference is reported 
as a mismatch.

The job completion log also reports mappers entered and mappers completed.

Tests
 * Unit tests for {{lastScannedKey}} versus {{{}endKey{}}}, the dummy progress 
marker, tag bytes in the digest, and the mapper skipping zero-row results.
 * {{{}testSyncTableValidateWithOnlyCellTagDifferences{}}}: identical cells 
that differ only by a tag mismatch on every row.
 * {{{}testSyncTableValidatePagingDummyRowSurfacedChunkSize10MB{}}}: dense 
tombstones with server paging enabled. The job finishes, counts only the live 
rows, and reports no mismatched chunks.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to