github-actions[bot] commented on code in PR #65490:
URL: https://github.com/apache/doris/pull/65490#discussion_r3565717206
##########
be/src/format_v2/column_mapper.cpp:
##########
@@ -553,6 +553,21 @@ static VExprSPtr rewrite_literal_to_file_type(const
VExprSPtr& literal_expr,
if (file_field.get_type() !=
remove_nullable(rewrite_info.file_type)->get_primitive_type()) {
return nullptr;
}
+ Field round_trip_field;
+ try {
+ convert_field_to_type(file_field, *original_literal->data_type(),
&round_trip_field,
+ rewrite_info.file_type.get());
+ } catch (const Exception&) {
+ return nullptr;
+ }
+ // Only localize a literal when converting it to the file type is
lossless. For example,
+ // BIGINT 1 -> INT 1 -> BIGINT 1 succeeds. However, for a table predicate
`value < 1.5`
+ // with DOUBLE table type and INT file type, rewriting 1.5 to INT 1 would
make a file value
+ // of 1 fail `value < 1` and silently drop a matching row. Its round trip
+ // DOUBLE 1.5 -> INT 1 -> DOUBLE 1.0 fails here and keeps the table-typed
predicate instead.
+ if (round_trip_field != original_field) {
Review Comment:
Round-trip equality is not enough to prove the file-local predicate is safe.
It still misses cases where the localized predicate is stricter than the final
table predicate. For complex literals, `convert_field_to_type()` recursively
converts array elements, but `Field::operator<=>` returns equal for
`TYPE_ARRAY`/`TYPE_MAP`/`TYPE_STRUCT` without comparing contents, so
`ARRAY<DOUBLE> [1.5] -> ARRAY<INT> [1] -> ARRAY<DOUBLE> [1.0]` is accepted and
can drop rows for supported array comparisons/IN predicates. For scalar casts
in the other direction, a table `INT` column backed by a file `DOUBLE` column
accepts `1 -> 1.0 -> 1`, so `value = 1` becomes `file_double = 1.0`; file value
`1.5` is filtered out even though the normal materialization path casts it to
table value `1` before the final scanner conjunct runs. Please restrict literal
localization to conversions that make the file predicate a true subset of the
table predicate, and skip or deeply compare complex literals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
