Ramin Gharib created FLINK-40978:
------------------------------------
Summary: Cast from VARIANT through FromVariantConverter
Key: FLINK-40978
URL: https://issues.apache.org/jira/browse/FLINK-40978
Project: Flink
Issue Type: Improvement
Components: Table SQL / Planner, Table SQL / Runtime
Affects Versions: 2.4.0
Reporter: Ramin Gharib
Assignee: Ramin Gharib
Every \{{CAST(v AS T)}} from \{{VARIANT}} generates its own code today. Six
planner cast rules each emit the walk for their target: primitives, ARRAY, ROW,
MAP and character strings, plus a shared base class. For \{{CAST(v AS
ARRAY<INT>)}} the generated code is:
{code:java}
int arraySize$2 = VariantCastUtils.arraySize(variant$1, "ARRAY<INT>");
java.lang.Integer[] objArray$3 = new java.lang.Integer[arraySize$2];
for (int i$4 = 0; i$4 < arraySize$2; i$4++) {
Variant element$5 = variant$1.getElement(i$4);
if (!element$5.isNull()) {
result$6 = ((int) VariantCastUtils.toIntegral(element$5, -2147483648L,
2147483647L, "INTEGER"));
objArray$3[i$4] = result$6;
}
}
result$0 = new GenericArrayData(objArray$3);
{code}
The write side went the other way in FLINK-40826. \{{CAST(x AS VARIANT)}} calls
a \{{ToVariantConverter}} in flink-table-common, built once per type. The read
side has no such class. A format that reads a \{{VARIANT}} into SQL types
cannot reuse the cast, and the null and shape rules are spread over several
rules.
h3. Proposal
Add \{{FromVariantConverter}} to flink-table-common as the twin of
\{{ToVariantConverter}}. It is built once per target type and returns internal
data. The generated code shrinks to one call:
{code:java}
result$1 = ((ArrayData)(fromVariantConverter$2.convert(variant$0)));
{code}
* Merge the six read-side rules into one \{{FromVariantCastRule}}. Printing
keeps its own path through \{{VariantCastUtils#toPrintString}}, because a
printed value must never fail.
* Move \{{VariantCastUtils}} from flink-table-runtime to flink-table-common.
The converter calls it, and common cannot depend on runtime. The class is
\{{@Internal}}.
* Add \{{getSessionZoneId()}} to \{{CodeGeneratorCastRule.Context}}. The
converter needs the zone when it is created. \{{CastRule.Context}} already
exposes it.
h3. Behavior
No user-visible change. Every result, error message and null rule stays the
same. Existing tests pass unchanged, apart from the rule names in
\{{CastRuleProviderTest}}.
Internal differences:
* A \{{ROW}} result is a \{{GenericRowData}} instead of a \{{BinaryRowData}},
because \{{BinaryRowWriter}} is not in common. A \{{VARIANT}} below a \{{ROW}}
is still copied out of its parent.
* An \{{ARRAY}} of a \{{NOT NULL}} primitive holds boxed values instead of a
primitive array.
* A primitive target boxes its value through the converter. The write side
accepted the same cost.
* A \{{DISTINCT}} target now casts like its source type. It used to fail during
code generation. SQL cannot create such a type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
