github-actions[bot] commented on code in PR #65446:
URL: https://github.com/apache/doris/pull/65446#discussion_r3709645039
##########
fe/be-java-extensions/hadoop-hudi-scanner/src/main/java/org/apache/doris/hudi/HadoopHudiColumnValue.java:
##########
@@ -133,7 +135,7 @@ public LocalDateTime getDateTime() {
return ((Timestamp) fieldData).toLocalDateTime();
} else if (fieldData instanceof TimestampWritableV2) {
return
LocalDateTime.ofInstant(Instant.ofEpochSecond((((TimestampObjectInspector)
fieldInspector)
- .getPrimitiveJavaObject(fieldData)).toEpochSecond()),
zoneId);
+ .getPrimitiveJavaObject(fieldData)).toEpochSecond()),
int96ZoneId);
Review Comment:
[P1] Scope this timezone conversion to Parquet INT96. A Hudi COW ORC base
file is routed through this JNI scanner when `force_jni_scanner=true`, and ORC
TIMESTAMP values arrive as `TimestampWritableV2` too. `HudiJniReader` supplies
`int96_time_zone` to every JNI split, while this decoder has no physical-format
discriminator, so an ORC scan with catalog and session zones that differ will
shift the JNI result even though the native ORC reader keeps ORC/session
semantics. Please carry the base physical format (or an explicit INT96 marker)
into this decoder, use the override only for Parquet INT96, and add
forced-JNI/native ORC parity coverage.
##########
be/src/format_v2/column_mapper.cpp:
##########
@@ -1803,6 +1842,7 @@ static Status add_scan_column(FileScanRequest*
file_request, ColumnMapping* mapp
// merge_filter_projection() adds `s -> b`, so the predicate column
reads both children.
RETURN_IF_ERROR(merge_filter_projection(filter_projections,
&projection));
}
+ attach_timestamp_semantics(*mapping, &projection);
Review Comment:
[P1] Preserve the timestamp semantic on children introduced only by a filter
projection. In a shape such as `SELECT s.payload WHERE s.ltz > ...`, the
visible mapping has no `ltz` child, so the fallback projection is built from
`original_file_children`; that helper copies only local IDs, and this later
walk can only annotate `mapping.child_mappings`. Paimon's high-precision
TIMESTAMP and TIMESTAMP_LTZ share unannotated INT96, and its native path
defaults `enable_mapping_timestamp_tz` off, so the merged `ltz` projection
stays DATETIMEV2 even though the history-annotated file-block child is
TIMESTAMPTZ. That violates the reader/type contract or evaluates the predicate
in the wrong domain. Please carry `timestamp_is_adjusted_to_utc` through
schema-fallback/merge nodes recursively and add a filter-only nested LTZ mapper
plus end-to-end predicate test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
