Aleksandr Efimov created IMPALA-15297:
-----------------------------------------
Summary: Calcite planner: map Impala operators onto Substrait
functions
Key: IMPALA-15297
URL: https://issues.apache.org/jira/browse/IMPALA-15297
Project: IMPALA
Issue Type: Sub-task
Components: Frontend
Reporter: Aleksandr Efimov
IMPALA-15296 added a test-scoped exporter for the pre-conversion logical plan.
Twenty of the twenty-four query shapes the direct harness covers export end to
end. The rest stop on an operator Isthmus cannot resolve.
Isthmus resolves a scalar call by looking the operator up in a map keyed on the
operator itself, and Calcite compares operators by class, name and kind. Impala
substitutes its own classes for a good number of functions, so an operator that
means exactly what Substrait's does still misses. Isthmus takes additional
signatures for this: they say which Substrait function an Impala operator
stands for and leave the operand matching alone.
Add the signatures for the operators that have a Substrait counterpart:
coalesce, concat, length, and floor and ceil with their d-prefixed spellings.
Impala reaches these by three separate routes, the USE_IMPALA_OPERATOR set in
ImpalaOperatorTable, builtins that Calcite has no operator for, and the
replacements in ImpalaCustomOperatorTable. All three resolve by name, so one
table covers them.
That takes the harness corpus from twenty to twenty-two. What is left needs a
different mechanism rather than another name. Substrait models a conditional as
the IfThen expression rather than as a function, so Impala's IF has no name to
map to, and lag is a window function resolved by a different converter.
Substrait's round takes its digit count as i32 where Impala types the literal
as i8, so the name lines up but the operands do not, and Substrait has no
truncate at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
