Hello Peter Rozsa, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24557
to look at the new patch set (#7).
Change subject: IMPALA-15057: Add variant_get() builtin and first-class VARIANT
expressions
......................................................................
IMPALA-15057: Add variant_get() builtin and first-class VARIANT expressions
The patch adds
variant_get(VARIANT, STRING) -> VARIANT
variant_get(VARIANT, STRING, STRING) -> <type>
try_variant_get() (2-args and 3-args as well)
variant_to_json(VARIANT).
There are two main parts: an infrastructure part that makes VARIANT a
first-class expression type, and a feature part that implements the
functions on top of it.
The infrastructure part makes VARIANT a first-class expression type. It
introduces the VariantVal UDF ABI (two StringVals: metadata + value) and
unblocks the mechanisms that previously restricted VARIANT to a
scan-only pass-through SlotRef: AnyVal sizing/alignment/marshalling,
symbol mangling, the Get*Val eval dispatch and interpreted overrides,
RawValue::WriteVariant materialization, and the FE type-resolution and
guard checks. Result rendering (hs2-util, query-result-set) now consumes
a VariantVal instead of a raw slot. Because VARIANT codegen is
unimplemented, any expression tree that involves a VARIANT type is
forced onto the interpreted path.
The feature part implements the functions in variant-functions-ir.cc:
* the 2-arg form returns a zero-copy slice (shared metadata, value
sub-slice)
* the 3-arg form coerces to primitive type with widening,
checked narrowing, and string parsing. STRING results are rendered
through the JSON serializer
* DATE is only produced from a DATE value, not silently reinterpreted
from an integer
The typed forms are registered under internal names because the catalog
cannot hold overloads that differ only by return type. FunctionCallExpr
maps the type tag to the internal overload.
Testing:
* backend tests added
* e2e tests added
Change-Id: I55bfed394ac2fb57135fadedd489f59e4cc10de4
Assisted-by: Claude Opus 4.8 (1M context) <[email protected]>
---
M be/src/exprs/CMakeLists.txt
M be/src/exprs/agg-fn-evaluator.cc
M be/src/exprs/anyval-util.h
M be/src/exprs/expr-value.h
M be/src/exprs/null-literal.cc
M be/src/exprs/null-literal.h
M be/src/exprs/scalar-expr-evaluator-ir.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/scalar-expr.cc
M be/src/exprs/scalar-expr.h
M be/src/exprs/scalar-expr.inline.h
M be/src/exprs/scalar-fn-call.cc
M be/src/exprs/scalar-fn-call.h
M be/src/exprs/slot-ref.cc
M be/src/exprs/slot-ref.h
M be/src/exprs/variant-functions-ir.cc
A be/src/exprs/variant-functions-test.cc
M be/src/exprs/variant-functions.h
M be/src/runtime/raw-value.cc
M be/src/runtime/variant-value.cc
M be/src/runtime/variant-value.h
M be/src/service/hs2-util.cc
M be/src/service/query-result-set.cc
M be/src/udf/udf-internal.h
M be/src/util/symbols-util.cc
M be/src/util/variant-util.cc
M be/src/util/variant-util.h
M common/function-registry/impala_functions.py
M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java
M fe/src/main/java/org/apache/impala/catalog/Function.java
M fe/src/main/java/org/apache/impala/catalog/VariantType.java
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-variant-get.test
M tests/query_test/test_iceberg.py
35 files changed, 1,827 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/24557/7
--
To view, visit http://gerrit.cloudera.org:8080/24557
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I55bfed394ac2fb57135fadedd489f59e4cc10de4
Gerrit-Change-Number: 24557
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
