github-actions[bot] commented on code in PR #68161:
URL: https://github.com/apache/doris/pull/68161#discussion_r4311761198
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java:
##########
@@ -238,19 +274,97 @@ private void initReader() throws IOException {
hasReadTypeProjection = true;
readFields.add(tableField.newType(
variantProjection.readType().copy(tableField.type().isNullable())));
+ currentFields.add(readFields.get(readFields.size() - 1));
}
outputIndex++;
}
+ boolean repairPolicy = requiresDatetimeV2PrecisionRepair && authSplit
!= null
+ && authSplit.authResult() != null;
+ List<DataField> outputReadFields = new ArrayList<>(readFields);
+ RowType outputType = new RowType(currentFields);
+ if (repairPolicy) {
+ PaimonTimestampQueryAuth.addPolicyFields(
+ authSplit.authResult(), policyTableType, readFields,
currentFields);
+ split = PaimonTimestampQueryAuth.repairPolicy(split, new
RowType(currentFields));
+ hasReadTypeProjection = true;
+ }
if (hasReadTypeProjection) {
readBuilder.withReadType(new RowType(readFields));
} else {
readBuilder.withProjection(projected);
}
+ // FE removes only predicates that depend on timestamp values being
repaired. Safe
+ // non-timestamp predicates remain useful for Paimon file and row
pruning.
readBuilder.withFilter(getPredicates());
- reader =
newReadWithOptionalIOManager(readBuilder).executeFilter().createReader(getSplit());
+ reader =
newReadWithOptionalIOManager(readBuilder).executeFilter().createReader(split);
+ if (repairPolicy) {
+ reader = PaimonTimestampQueryAuth.projectOutput(reader, new
RowType(currentFields), outputType);
+ }
// Decode with the types that were REQUESTED, not the table's: a
pruned nested column comes back
// narrow, and taking table.rowType() here would misalign every one of
them.
- paimonDataTypeList =
readFields.stream().map(DataField::type).collect(Collectors.toList());
+ paimonDataTypeList =
outputReadFields.stream().map(DataField::type).collect(Collectors.toList());
+ }
+
+ private static Table dataSource(Table source) {
+ if (source instanceof FileStoreTable) {
+ return unwrapSystemPlanningSource((FileStoreTable) source);
+ }
+ if (!(source instanceof AuditLogTable || source instanceof
ReadOptimizedTable
+ || source instanceof RowTrackingTable)) {
+ return source;
+ }
+ // A row-backed system wrapper synthesizes its system fields outside
the data reader.
+ // Resolve the pinned data source, without reopening the catalog.
+ Optional<FileStoreTable> systemSource = hiddenSystemSource(source);
+ return systemSource.isPresent() ? dataSource(systemSource.get()) :
source;
+ }
+
+ private static RowType policyRowType(Table source, Split split) {
+ source = dataSource(source);
+ if (source instanceof FallbackReadFileStoreTable) {
+ FallbackReadFileStoreTable pair = (FallbackReadFileStoreTable)
source;
+ boolean readFallback = split instanceof FallbackSplit &&
((FallbackSplit) split).isFallback();
+ FileStoreTable branch = isWrappedFirst(pair) != readFallback ?
pair.wrapped() : pair.other();
Review Comment:
[P1] Resolve ChainSplit fields from the delta reader schema
`ChainGroupReadTable` is also a `FallbackReadFileStoreTable`, but its
[reader always dispatches a `ChainSplit` to
`other()`](https://github.com/apache/paimon/blob/release-1.4.2/paimon-core/src/main/java/org/apache/paimon/table/ChainGroupReadTable.java#L530-L532).
Here that split has no `FallbackSplit` flag, so recursion instead selects the
snapshot schema. This is reachable through `t$audit_log`: FE excludes system
tables from its ordinary chain-table rejection and sends these splits to JNI.
For primary-key branches keyed by `(pt,id)` and partitioned by `pt`, with
current `ts TIMESTAMP(0)`, main/delta `ts#2`, snapshot `ts#3` after
drop/re-add, and a delta-only partition retaining a p6 file, repair rebinds the
requested timestamp to #3 although the delta file stores #2. A stored non-NULL
whole-second value then becomes NULL, whereas the previous outer-schema #2
projection could read it. Handle the chain reader's dispatch before the generic
fallback recursion and add a historical au
dit-log reader case with independent snapshot/delta IDs. This is a separate
subtype-dispatch instance from the earlier ordinary fallback-pair finding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
