OIiveirra commented on code in PR #68161:
URL: https://github.com/apache/doris/pull/68161#discussion_r4053098973
##########
be/src/format_v2/column_mapper.cpp:
##########
@@ -2898,6 +2901,12 @@ Status TableColumnMapper::_create_direct_mapping(const
ColumnDefinition& table_c
mapping->projected_file_children = file_field.children;
mapping->timestamp_is_adjusted_to_utc =
file_field.timestamp_is_adjusted_to_utc;
mapping->file_type = file_field.type;
+ const auto file_type = remove_nullable(mapping->file_type);
+ const auto table_type = remove_nullable(mapping->table_type);
+ mapping->truncate_datetimev2_precision =
_options.truncate_datetimev2_precision_for_paimon &&
+ file_type->get_primitive_type()
== TYPE_DATETIMEV2 &&
Review Comment:
Fixed in 5f4e7d2b95e. The Paimon-only narrowing rule now applies to
same-primitive TIMESTAMPTZ mappings, forces final projection/filter evaluation,
and truncates ColumnTimeStampTz fractional microseconds; added a p6-to-p0
mapper test. Ordinary SQL CAST behavior is unchanged.
##########
be/src/format/column_type_convert.cpp:
##########
@@ -373,6 +373,11 @@ std::unique_ptr<ColumnTypeConverter>
ColumnTypeConverter::get_converter(const Da
return _decimal_converter(src_type, dst_type);
}
+ if (src_primitive_type == TYPE_DATETIMEV2 && dst_primitive_type ==
TYPE_DATETIMEV2 &&
+ src_type->get_scale() > dst_type->get_scale()) {
+ return
std::make_unique<DateTimeV2PrecisionConverter>(dst_type->get_scale());
Review Comment:
Fixed in 5f4e7d2b95e. V1 now disables Parquet metadata/page and ORC SARG
predicate pushdown when Paimon narrows timestamp precision; added Parquet/ORC
equality and range regression coverage.
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java:
##########
@@ -167,7 +168,8 @@ public LocalDate getDate() {
@Override
public LocalDateTime getDateTime() {
- Timestamp ts = record.getTimestamp(idx, dorisType.getPrecision());
+ Timestamp ts = DateTimeUtils.truncate(
Review Comment:
Fixed in 5f4e7d2b95e. Replaced Paimon's helper with fixed-width fractional
truncation and added coverage for .000001 at p4 plus pre-epoch .600 to p0.
##########
be/src/format_v2/expr/cast.cpp:
##########
@@ -116,6 +148,13 @@ Status Cast::_do_execute(VExprContext* context, const
Block* block, const Select
ColumnPtr tmp_arg_column;
RETURN_IF_ERROR(_children[0]->execute_column(context, block, selector,
count, tmp_arg_column));
auto arg_type = _children[0]->execute_type(block);
+ if (_truncate_datetimev2_precision &&
can_truncate_datetimev2_precision(arg_type, _data_type)) {
+ auto result = _data_type->create_column();
+ result->insert_range_from(*tmp_arg_column, 0, count);
Review Comment:
Fixed in 5f4e7d2b95e. The precision-conversion copy now materializes the
target nullable wrapper and populates a non-null map for required file columns;
added a required-to-nullable mapper test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
