lilei1128 commented on code in PR #10415:
URL: https://github.com/apache/paimon/pull/10415#discussion_r4219173484
##########
paimon-core/src/main/java/org/apache/paimon/operation/DataEvolutionSplitRead.java:
##########
@@ -1223,16 +1231,25 @@ static boolean shouldReadRowSidecar(
@Nullable List<Range> rowRanges,
long maxSelectedRows,
double maxSelectionRatio) {
- if (rowRanges == null
- || rowRanges.isEmpty()
+ return shouldReadRowSidecar(file, rowRanges, null, maxSelectedRows,
maxSelectionRatio);
+ }
+
+ @VisibleForTesting
+ static boolean shouldReadRowSidecar(
+ DataFileMeta file,
+ @Nullable List<Range> rowRanges,
+ @Nullable FileIndexResult fileIndexResult,
+ long maxSelectedRows,
+ double maxSelectionRatio) {
+ if (isNullOrEmpty(rowRanges)
|| file.rowCount() <= 0
|| isBlobFile(file.fileName())
|| isVectorStoreFile(file.fileName())
|| rowSidecarFileName(file) == null) {
return false;
}
- long selectedRowCount = selectedRowCount(file, rowRanges);
+ long selectedRowCount = selectedRowCount(file, rowRanges,
fileIndexResult);
Review Comment:
Fixed by using the physical data schema for row-sidecar decoding, while
continuing to supply row-tracking fields from the manifest. Added regression
coverage for full explicit row ranges and _ROW_ID predicates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
