Aleksandr Efimov created IMPALA-15296:
-----------------------------------------
Summary: Calcite planner: export the pre-conversion logical plan
to Substrait
Key: IMPALA-15296
URL: https://issues.apache.org/jira/browse/IMPALA-15296
Project: IMPALA
Issue Type: Sub-task
Components: Frontend
Reporter: Aleksandr Efimov
IMPALA-15282 added a test-only compatibility test that converts the
pre-conversion logical plan with Isthmus. It answers whether a handful of
covered shapes still convert. It does not give a caller a way to ask for the
conversion, or a usable answer when the conversion fails.
Add a test-scoped exporter over the same seam. It takes the plan and the
client-visible labels and returns either a Substrait plan or a rejection
carrying a reason, what Isthmus said, and the relations or operators the plan
contained. A half-converted plan is never returned.
What Isthmus accepts is decided by Isthmus rather than by a list of operators
kept in Impala. Such a list would drift as the operator table changes, and it
would also be wrong: Impala's aggregate classes convert because Isthmus
normalises aggregates by SqlKind, while its scalar functions resolve on
operator identity and do not, so the class name says nothing about whether an
operator exports.
Relations are the exception and are checked before conversion, because Impala's
own relations are unambiguously outside Substrait and listing them is more use
than the exception Isthmus raises on the first one it meets. That is the
cte_threshold case: above the threshold the repeated subtree becomes an
ImpalaSequence over a producer and one consumer per reference, and the seam
stops being a plan of Calcite relations.
The client-visible labels are passed in rather than read off the plan, because
the seam's row type is upper-cased and exporting its names would rename the
client's columns.
Isthmus stays test-scoped: no production dependency, no query option, nothing
added to impala-package.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
