lilei1128 opened a new pull request, #10792:
URL: https://github.com/apache/paimon/pull/10792
### Purpose
Row sidecar selection estimated the selected row count from the requested
row ranges and, when available, a bitmap file-index result. A deletion
vector influenced that estimate only indirectly: the caller had already
merged it into the bitmap result, and only for 32-bit deletion vectors.
Without a file index, or with a 64-bit deletion vector, deleted rows were
therefore counted as live, and the heuristic could reject a row sidecar
that would actually have paid off.
Pass the group's DeletionVectorWithRange down to
shouldReadRowSidecar/selectedRowCount and compute the live row count
explicitly:
- use rowCount - getCardinality() when the deletion vector covers exactly
the file, avoiding a full scan;
- otherwise scan the smaller of the selection and the deletion vector;
- keep the existing behavior when the deletion vector is empty.
Applying the deletion vector on top of an already intersected file-index
result is idempotent, so the read-path preprocessing and the actual reads
are unchanged, while both 32-bit and 64-bit deletion vectors are now
handled. The change only affects the sidecar heuristic; rows are still
filtered by ApplyDeletionVectorReader at read time.
### Tests
Add unit tests in DataEvolutionSplitReadTest covering:
- no file index: live rows after deletion drive the decision;
- anchor-range offset between the file and the deletion vector range,
including positions outside the file;
- the sparse deletion vector scan when the selection is larger than the
deletion vector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
