This is an automated email from the ASF dual-hosted git repository.
snuyanzin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 5d3597c253f [FLINK-37926][table] Support casting from `VARIANT` to
`ARRAY`
5d3597c253f is described below
commit 5d3597c253fcb6383ca74df9c43257c283dc821e
Author: Ramin Gharib <[email protected]>
AuthorDate: Fri Sep 4 14:35:59 2026 +0200
[FLINK-37926][table] Support casting from `VARIANT` to `ARRAY`
---
docs/content.zh/docs/sql/reference/data-types.md | 28 +-
docs/content/docs/sql/reference/data-types.md | 28 +-
.../types/logical/utils/LogicalTypeCasts.java | 8 +
.../flink/table/types/LogicalTypeCastsTest.java | 27 +-
.../AbstractVariantToConstructedCastRule.java | 71 ++++
.../functions/casting/CastRuleProvider.java | 1 +
.../functions/casting/VariantToArrayCastRule.java | 171 +++++++++
.../planner/functions/CastFunctionITCase.java | 121 ++++++-
.../functions/casting/CastRuleProviderTest.java | 21 ++
.../planner/functions/casting/CastRulesTest.java | 384 ++++++++++++++-------
.../table/runtime/functions/VariantCastUtils.java | 21 ++
11 files changed, 741 insertions(+), 140 deletions(-)
diff --git a/docs/content.zh/docs/sql/reference/data-types.md
b/docs/content.zh/docs/sql/reference/data-types.md
index 62266a004eb..b1e938567f2 100644
--- a/docs/content.zh/docs/sql/reference/data-types.md
+++ b/docs/content.zh/docs/sql/reference/data-types.md
@@ -1588,6 +1588,32 @@ rendering. Use `JSON_STRING` for the JSON representation
instead, where a string
`"foo"` and an object or array is serialized. A variant that stores a JSON
`null` casts to SQL
`NULL`.
+A `VARIANT` can also be cast to a constructed target, which imposes a schema
on it. A variant array
+casts to `ARRAY<T>`. The variant must be an array, otherwise the cast fails.
Each element is itself a
+`VARIANT`, so it casts to the element type `T` by the same rules, recursively.
A leaf is never parsed
+either, so a stored string does not reach an integer target. Cast the leaf to
`STRING` first and
+convert with a regular cast.
+
+- Each element casts to `T`. A variant null element maps to SQL `NULL` when
`T` is nullable and fails
+ the cast when `T` is `NOT NULL`. An empty array casts to an empty `ARRAY<T>`.
+- `ARRAY<VARIANT>` is the identity element: it shreds one level and keeps each
element as a variant. A
+ null element stays a variant null rather than becoming SQL `NULL`.
+
+If any element cast fails, the whole cast fails, and `TRY_CAST` returns `NULL`
for the entire value
+rather than a partial result. An element type with no variant counterpart,
such as
+`ARRAY<INTERVAL YEAR TO MONTH>`, is rejected at validation.
+
+The following examples use `a` for `PARSE_JSON('[1, 2, 3]')` and `m` for the
mixed array
+`PARSE_JSON('[1, "a"]')`:
+
+```sql
+CAST(a AS ARRAY<INT>) -- [1, 2, 3]
+CAST(a AS ARRAY<STRING>) -- ['1', '2', '3'], each element rendered like the
scalar cast
+CAST(m AS ARRAY<INT>) -- fails on "a", a stored string is not parsed into
an integer
+CAST(m AS ARRAY<STRING>) -- ['1', 'a'], a heterogeneous array still renders
each element
+CAST(m AS ARRAY<VARIANT>) -- [1, "a"] as variants, one level shredded
+```
+
**Declaration**
{{< tabs "25c30432-8460-441d-a036-9416d8202882" >}}
@@ -1802,7 +1828,7 @@ COALESCE(TRY_CAST('non-number' AS INT), 0) --- 结果返回数字 0 的
INT 格
| `ROW` | Y
| N | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | !³
| N | N | N | N |
| `STRUCTURED` | Y
| N | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| !³ | N | N | N |
| `RAW` | Y
| ! | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| N | Y⁴ | N | N |
-| `VARIANT` | N
| ! | ! | ! | !
| ! | ! | ! | ! | ! | ! | N |
! | ! | N | N | N | N | N
| N | N | Y | N |
+| `VARIANT` | N
| ! | ! | ! | !
| ! | ! | ! | ! | ! | ! | N |
! | ! | N | !³ | N | N | N
| N | N | Y | N |
| `BITMAP` | Y
| Y⁷ | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| N | N | N | N |
备注：
diff --git a/docs/content/docs/sql/reference/data-types.md
b/docs/content/docs/sql/reference/data-types.md
index 207c83d2792..442536c212e 100644
--- a/docs/content/docs/sql/reference/data-types.md
+++ b/docs/content/docs/sql/reference/data-types.md
@@ -1596,6 +1596,32 @@ rendering. Use `JSON_STRING` for the JSON representation
instead, where a string
`"foo"` and an object or array is serialized. A variant that stores a JSON
`null` casts to SQL
`NULL`.
+A `VARIANT` can also be cast to a constructed target, which imposes a schema
on it. A variant array
+casts to `ARRAY<T>`. The variant must be an array, otherwise the cast fails.
Each element is itself a
+`VARIANT`, so it casts to the element type `T` by the same rules, recursively.
A leaf is never parsed
+either, so a stored string does not reach an integer target. Cast the leaf to
`STRING` first and
+convert with a regular cast.
+
+- Each element casts to `T`. A variant null element maps to SQL `NULL` when
`T` is nullable and fails
+ the cast when `T` is `NOT NULL`. An empty array casts to an empty `ARRAY<T>`.
+- `ARRAY<VARIANT>` is the identity element: it shreds one level and keeps each
element as a variant. A
+ null element stays a variant null rather than becoming SQL `NULL`.
+
+If any element cast fails, the whole cast fails, and `TRY_CAST` returns `NULL`
for the entire value
+rather than a partial result. An element type with no variant counterpart,
such as
+`ARRAY<INTERVAL YEAR TO MONTH>`, is rejected at validation.
+
+The following examples use `a` for `PARSE_JSON('[1, 2, 3]')` and `m` for the
mixed array
+`PARSE_JSON('[1, "a"]')`:
+
+```sql
+CAST(a AS ARRAY<INT>) -- [1, 2, 3]
+CAST(a AS ARRAY<STRING>) -- ['1', '2', '3'], each element rendered like the
scalar cast
+CAST(m AS ARRAY<INT>) -- fails on "a", a stored string is not parsed into
an integer
+CAST(m AS ARRAY<STRING>) -- ['1', 'a'], a heterogeneous array still renders
each element
+CAST(m AS ARRAY<VARIANT>) -- [1, "a"] as variants, one level shredded
+```
+
**Declaration**
{{< tabs "25c30432-8460-441d-a036-9416d8202882" >}}
@@ -1811,7 +1837,7 @@ The matrix below describes the supported cast pairs,
where "Y" means supported,
| `ROW` | Y
| N | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | !³
| N | N | N | N |
| `STRUCTURED` | Y
| N | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| !³ | N | N | N |
| `RAW` | Y
| ! | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| N | Y⁴ | N | N |
-| `VARIANT` | N
| ! | ! | ! | !
| ! | ! | ! | ! | ! | ! | N |
! | ! | N | N | N | N | N
| N | N | Y | N |
+| `VARIANT` | N
| ! | ! | ! | !
| ! | ! | ! | ! | ! | ! | N |
! | ! | N | !³ | N | N | N
| N | N | Y | N |
| `BITMAP` | Y
| Y⁷ | N | N | N
| N | N | N | N | N | N | N |
N | N | N | N | N | N | N
| N | N | N | N |
Notes:
diff --git
a/flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java
b/flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java
index d22b16399c3..2ab51ba1098 100644
---
a/flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java
+++
b/flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java
@@ -19,6 +19,7 @@
package org.apache.flink.table.types.logical.utils;
import org.apache.flink.annotation.Internal;
+import org.apache.flink.table.types.logical.ArrayType;
import org.apache.flink.table.types.logical.DateType;
import org.apache.flink.table.types.logical.DayTimeIntervalType;
import org.apache.flink.table.types.logical.DistinctType;
@@ -53,6 +54,7 @@ import static
org.apache.flink.table.types.logical.LogicalTypeFamily.NUMERIC;
import static
org.apache.flink.table.types.logical.LogicalTypeFamily.PREDEFINED;
import static org.apache.flink.table.types.logical.LogicalTypeFamily.TIME;
import static org.apache.flink.table.types.logical.LogicalTypeFamily.TIMESTAMP;
+import static org.apache.flink.table.types.logical.LogicalTypeRoot.ARRAY;
import static org.apache.flink.table.types.logical.LogicalTypeRoot.BIGINT;
import static org.apache.flink.table.types.logical.LogicalTypeRoot.BINARY;
import static org.apache.flink.table.types.logical.LogicalTypeRoot.BITMAP;
@@ -666,6 +668,12 @@ public final class LogicalTypeCasts {
return supportsStructuredCasting(
sourceType, targetType, (s, t) -> supportsCasting(s, t,
allowExplicit));
+ } else if (sourceRoot == VARIANT && targetRoot == ARRAY) {
+ // A variant array casts to ARRAY<T> when VARIANT casts to the
single element type T.
+ // Explicit only, so no accidental coercion. Each runtime element
is cast to T by the
+ // array cast rule; a per-element mismatch fails there, not here.
+ return allowExplicit
+ && supportsCasting(sourceType, ((ArrayType)
targetType).getElementType(), true);
} else if (sourceRoot == RAW
&& !targetType.is(BINARY_STRING)
&& !targetType.is(CHARACTER_STRING)
diff --git
a/flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java
b/flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java
index c51408edd81..07b0f920911 100644
---
a/flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java
+++
b/flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java
@@ -34,7 +34,7 @@ import org.apache.flink.table.types.logical.FloatType;
import org.apache.flink.table.types.logical.IntType;
import org.apache.flink.table.types.logical.LocalZonedTimestampType;
import org.apache.flink.table.types.logical.LogicalType;
-import org.apache.flink.table.types.logical.MapType;
+import org.apache.flink.table.types.logical.MultisetType;
import org.apache.flink.table.types.logical.NullType;
import org.apache.flink.table.types.logical.RawType;
import org.apache.flink.table.types.logical.RowType;
@@ -59,7 +59,6 @@ import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.Arrays;
-import java.util.List;
import java.util.stream.Stream;
import static org.assertj.core.api.Assertions.assertThat;
@@ -289,13 +288,29 @@ class LogicalTypeCastsTest {
Arguments.of(new VariantType(), VarCharType.STRING_TYPE,
false, true),
// variant identity cast is implicit
Arguments.of(new VariantType(), new VariantType(), true, true),
- // TIME and constructed targets are not castable from variant
+ // TIME has no variant counterpart, so it is not castable from
variant
Arguments.of(new VariantType(), new TimeType(), false, false),
- Arguments.of(new VariantType(), new ArrayType(new IntType()),
false, false),
- Arguments.of(new VariantType(), new RowType(List.of()), false,
false),
+ // A variant imposes a schema on a constructed target,
explicit only, recursing on
+ // every leaf, which is itself a VARIANT cast
+ Arguments.of(new VariantType(), new ArrayType(new IntType()),
false, true),
+ Arguments.of(new VariantType(), new ArrayType(new
VariantType()), false, true),
Arguments.of(
new VariantType(),
- new MapType(new IntType(), new CharType()),
+ new ArrayType(new ArrayType(new IntType())),
+ false,
+ true),
+ // A leaf with no variant counterpart makes the whole
constructed cast unsupported
+ Arguments.of(
+ new VariantType(),
+ new ArrayType(
+ new YearMonthIntervalType(
+
YearMonthIntervalType.YearMonthResolution.MONTH)),
+ false,
+ false),
+ // MULTISET has no variant counterpart and stays unsupported
+ Arguments.of(
+ new VariantType(),
+ new MultisetType(VarCharType.STRING_TYPE),
false,
false));
}
diff --git
a/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/AbstractVariantToConstructedCastRule.java
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/AbstractVariantToConstructedCastRule.java
new file mode 100644
index 00000000000..f003b129ec2
--- /dev/null
+++
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/AbstractVariantToConstructedCastRule.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions.casting;
+
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.LogicalTypeRoot;
+import org.apache.flink.types.variant.Variant;
+
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.methodCall;
+
+/**
+ * Base class for the rules that cast a {@link LogicalTypeRoot#VARIANT} to a
constructed target,
+ * imposing a schema on a variant. A constructed cast is the scalar cast
applied to every leaf plus
+ * a shape check at each level, so the recursion bottoms out at the same
scalar cast the primitive
+ * and string rules perform and no new leaf semantics are introduced.
+ *
+ * <p>A constructed cast can always fail, on a shape mismatch, an unreadable
leaf, or a missing
+ * {@code NOT NULL} field, so {@code TRY_CAST} wraps the whole value and
returns {@code NULL} for
+ * any failure rather than a partial result.
+ */
+abstract class AbstractVariantToConstructedCastRule<OUT>
+ extends AbstractNullAwareCodeGeneratorCastRule<Variant, OUT> {
+
+ protected AbstractVariantToConstructedCastRule(CastRulePredicate
predicate) {
+ super(predicate);
+ }
+
+ @Override
+ public boolean canFail(LogicalType inputLogicalType, LogicalType
targetLogicalType) {
+ return true;
+ }
+
+ /**
+ * Treats a variant that stores a {@code null} as a {@code NULL} input, so
a top-level VARIANT
+ * null casts to SQL {@code NULL} before any shape check runs. Only
applied for a nullable
+ * target: a {@code NOT NULL} result cannot carry {@code NULL}, so a
null-valued variant then
+ * fails the shape check as a regular mismatch.
+ */
+ @Override
+ public CastCodeBlock generateCodeBlock(
+ CodeGeneratorCastRule.Context context,
+ String inputTerm,
+ String inputIsNullTerm,
+ LogicalType inputLogicalType,
+ LogicalType targetLogicalType) {
+ if (!targetLogicalType.isNullable()) {
+ return super.generateCodeBlock(
+ context, inputTerm, inputIsNullTerm, inputLogicalType,
targetLogicalType);
+ }
+ final String isNullTerm =
+ "(" + inputIsNullTerm + " || " + methodCall(inputTerm,
"isNull") + ")";
+ return super.generateCodeBlock(
+ context, inputTerm, isNullTerm, inputLogicalType,
targetLogicalType);
+ }
+}
diff --git
a/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleProvider.java
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleProvider.java
index 133542ec6f5..2b2cfc0e056 100644
---
a/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleProvider.java
+++
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleProvider.java
@@ -98,6 +98,7 @@ public class CastRuleProvider {
// Variant rules
.addRule(VariantToStringCastRule.INSTANCE)
.addRule(VariantToPrimitiveCastRule.INSTANCE)
+ .addRule(VariantToArrayCastRule.INSTANCE)
// Bitmap rules
.addRule(BitmapToStringCastRule.INSTANCE)
.addRule(BitmapToBinaryCastRule.INSTANCE)
diff --git
a/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/VariantToArrayCastRule.java
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/VariantToArrayCastRule.java
new file mode 100644
index 00000000000..5761cb55044
--- /dev/null
+++
b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/VariantToArrayCastRule.java
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions.casting;
+
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.GenericArrayData;
+import org.apache.flink.table.planner.codegen.CodeGenUtils;
+import org.apache.flink.table.runtime.functions.VariantCastUtils;
+import org.apache.flink.table.types.logical.ArrayType;
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.LogicalTypeRoot;
+import org.apache.flink.types.variant.Variant;
+
+import static org.apache.flink.table.planner.codegen.CodeGenUtils.newName;
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.constructorCall;
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.methodCall;
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.newArray;
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.staticCall;
+import static
org.apache.flink.table.planner.functions.casting.CastRuleUtils.strLiteral;
+
+/**
+ * {@link LogicalTypeRoot#VARIANT} to {@link LogicalTypeRoot#ARRAY} cast rule.
+ *
+ * <p>The variant must be an array, otherwise the cast fails. Each element is
itself a variant and
+ * casts to the target element type by the full {@code VARIANT}-to-element
rule, recursively. An
+ * element that stores a {@code null} maps to SQL {@code NULL} when the
element type is nullable and
+ * fails the cast when it is {@code NOT NULL}. The exception is a {@code
VARIANT} element type:
+ * there the element cast is the identity, so a VARIANT null element is kept
as a variant null
+ * rather than downgraded to SQL {@code NULL}.
+ */
+class VariantToArrayCastRule extends
AbstractVariantToConstructedCastRule<ArrayData> {
+
+ static final VariantToArrayCastRule INSTANCE = new
VariantToArrayCastRule();
+
+ private VariantToArrayCastRule() {
+ super(
+ CastRulePredicate.builder()
+ .predicate(
+ (input, target) ->
+ input.is(LogicalTypeRoot.VARIANT)
+ &&
target.is(LogicalTypeRoot.ARRAY)
+ && CastRuleProvider.resolve(
+ input,
+ ((ArrayType)
target)
+
.getElementType())
+ != null)
+ .build());
+ }
+
+ /* Example generated code for ARRAY<INT>:
+
+ int arraySize$2 =
+
org.apache.flink.table.runtime.functions.VariantCastUtils.arraySize(
+ variant$1, "ARRAY<INT>");
+ java.lang.Integer[] objArray$3 = new java.lang.Integer[arraySize$2];
+ for (int i$4 = 0; i$4 < arraySize$2; i$4++) {
+ org.apache.flink.types.variant.Variant element$5 =
variant$1.getElement(i$4);
+ if (!element$5.isNull()) {
+ result$6 =
+ ((int)
org.apache.flink.table.runtime.functions.VariantCastUtils.toIntegral(
+ element$5, -2147483648L, 2147483647L, "INTEGER"));
+ objArray$3[i$4] = result$6;
+ }
+ }
+ result$0 = new org.apache.flink.table.data.GenericArrayData(objArray$3);
+
+ A VARIANT null element leaves the slot null (SQL NULL); a NOT NULL element
type throws instead.
+
+ */
+ @Override
+ protected String generateCodeBlockInternal(
+ CodeGeneratorCastRule.Context context,
+ String inputTerm,
+ String returnVariable,
+ LogicalType inputLogicalType,
+ LogicalType targetLogicalType) {
+ final LogicalType elementType = ((ArrayType)
targetLogicalType).getElementType();
+ final String elementTypeTerm = arrayElementType(elementType);
+ final String sizeTerm = newName(context.getCodeGeneratorContext(),
"arraySize");
+ final String arrayTerm = newName(context.getCodeGeneratorContext(),
"objArray");
+ final String elementTerm = newName(context.getCodeGeneratorContext(),
"element");
+
+ // For a typed element the VARIANT null is handled in the loop below,
so the inner cast is
+ // the plain VARIANT-to-element rule on a non-null variant: a nullable
element maps it to
+ // SQL NULL, a NOT NULL element fails. For a VARIANT element the inner
cast is the identity,
+ // which keeps a variant null as-is.
+ final CastCodeBlock elementCast =
+ CastRuleProvider.generateAlwaysNonNullCodeBlock(
+ context, elementTerm, inputLogicalType, elementType);
+
+ return new CastRuleUtils.CodeWriter()
+ .declStmt(
+ int.class,
+ sizeTerm,
+ staticCall(
+ VariantCastUtils.class,
+ "arraySize",
+ inputTerm,
+
strLiteral(targetLogicalType.asSummaryString())))
+ .declStmt(elementTypeTerm + "[]", arrayTerm,
newArray(elementTypeTerm, sizeTerm))
+ .forStmt(
+ sizeTerm,
+ (index, loopWriter) -> {
+ loopWriter.declStmt(
+ Variant.class,
+ elementTerm,
+ methodCall(inputTerm, "getElement",
index));
+ if (elementType.is(LogicalTypeRoot.VARIANT)) {
+ // The element cast is the identity, so a
VARIANT null element is a
+ // valid variant null and is kept as-is rather
than downgraded to
+ // SQL NULL, matching the top-level VARIANT
cast.
+ loopWriter
+ .append(elementCast)
+ .assignArrayStmt(
+ arrayTerm, index,
elementCast.getReturnTerm());
+ return;
+ }
+ final String isPresent = "!" +
methodCall(elementTerm, "isNull");
+ if (elementType.isNullable()) {
+ loopWriter.ifStmt(
+ isPresent,
+ thenWriter ->
+ thenWriter
+ .append(elementCast)
+ .assignArrayStmt(
+ arrayTerm,
+ index,
+
elementCast.getReturnTerm()));
+ } else {
+ loopWriter.ifStmt(
+ isPresent,
+ thenWriter ->
+ thenWriter
+ .append(elementCast)
+ .assignArrayStmt(
+ arrayTerm,
+ index,
+
elementCast.getReturnTerm()),
+ elseWriter ->
+ elseWriter.throwStmt(
+ "new
org.apache.flink.table.api.TableRuntimeException(\"Cannot cast a VARIANT null
array element to a NOT NULL element type.\")"));
+ }
+ },
+ context.getCodeGeneratorContext())
+ .assignStmt(returnVariable,
constructorCall(GenericArrayData.class, arrayTerm))
+ .toString();
+ }
+
+ private static String arrayElementType(LogicalType elementType) {
+ if (elementType.isNullable()) {
+ return CodeGenUtils.boxedTypeTermForType(elementType);
+ }
+ return CodeGenUtils.primitiveTypeTermForType(elementType);
+ }
+}
diff --git
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java
index aee2d70c1cf..083fda47dff 100644
---
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java
+++
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java
@@ -76,6 +76,7 @@ import static
org.apache.flink.table.api.DataTypes.TIMESTAMP_LTZ;
import static org.apache.flink.table.api.DataTypes.TINYINT;
import static org.apache.flink.table.api.DataTypes.VARBINARY;
import static org.apache.flink.table.api.DataTypes.VARCHAR;
+import static org.apache.flink.table.api.DataTypes.VARIANT;
import static org.apache.flink.table.api.DataTypes.YEAR;
import static org.apache.flink.table.api.Expressions.$;
import static org.apache.flink.table.api.Expressions.call;
@@ -138,9 +139,13 @@ public class CastFunctionITCase extends
BuiltInFunctionTestBase {
}
private static List<TestSetSpec> variantCasts() {
- // A variant is produced with PARSE_JSON since there is no VARIANT
literal. Numeric casts
- // succeed only when the value is preserved exactly, otherwise CAST
fails and TRY_CAST
- // returns NULL.
+ final List<TestSetSpec> specs = new ArrayList<>();
+ specs.addAll(variantPrimitiveCasts());
+ specs.addAll(variantArrayCasts());
+ return specs;
+ }
+
+ private static List<TestSetSpec> variantPrimitiveCasts() {
return List.of(
TestSetSpec.forExpression("Cast a VARIANT produced by
PARSE_JSON to a primitive")
.onFieldsWithData("unused")
@@ -379,6 +384,116 @@ public class CastFunctionITCase extends
BuiltInFunctionTestBase {
TINYINT()));
}
+ private static List<TestSetSpec> variantArrayCasts() {
+ return List.of(
+ TestSetSpec.forExpression("Cast a VARIANT produced by
PARSE_JSON to an ARRAY")
+ .onFieldsWithData("unused")
+ .andDataTypes(STRING())
+ // ARRAY: each element casts by the same
VARIANT-to-element rule.
+ .testResult(
+ call("PARSE_JSON", "[1, 2,
3]").cast(ARRAY(INT())),
+ "CAST(PARSE_JSON('[1, 2, 3]') AS ARRAY<INT>)",
+ new Integer[] {1, 2, 3},
+ ARRAY(INT()).notNull())
+ // an approximate leaf takes any numeric kind
+ .testResult(
+ call("PARSE_JSON", "[1, 2,
3]").cast(ARRAY(DOUBLE())),
+ "CAST(PARSE_JSON('[1, 2, 3]') AS
ARRAY<DOUBLE>)",
+ new Double[] {1.0, 2.0, 3.0},
+ ARRAY(DOUBLE()).notNull())
+ // each element renders to string like the scalar cast
+ .testResult(
+ call("PARSE_JSON", "[1, 2,
3]").cast(ARRAY(STRING())),
+ "CAST(PARSE_JSON('[1, 2, 3]') AS
ARRAY<STRING>)",
+ new String[] {"1", "2", "3"},
+ ARRAY(STRING()).notNull())
+ // a heterogeneous array renders every element to
string
+ .testResult(
+ call("PARSE_JSON", "[1, \"a\", 2,
\"b\"]").cast(ARRAY(STRING())),
+ "CAST(PARSE_JSON('[1, \"a\", 2, \"b\"]') AS
ARRAY<STRING>)",
+ new String[] {"1", "a", "2", "b"},
+ ARRAY(STRING()).notNull())
+ .testResult(
+ call("PARSE_JSON", "[]").cast(ARRAY(INT())),
+ "CAST(PARSE_JSON('[]') AS ARRAY<INT>)",
+ new Integer[] {},
+ ARRAY(INT()).notNull())
+ // a VARIANT null element maps to SQL NULL for a
nullable element type
+ .testResult(
+ call("PARSE_JSON", "[1, null,
3]").cast(ARRAY(INT())),
+ "CAST(PARSE_JSON('[1, null, 3]') AS
ARRAY<INT>)",
+ new Integer[] {1, null, 3},
+ ARRAY(INT()).notNull())
+ // a VARIANT null element fails a NOT NULL element type
+ .testTableApiRuntimeError(
+ call("PARSE_JSON", "[1, null,
3]").cast(ARRAY(INT().notNull())),
+ "NOT NULL element type")
+ .testSqlRuntimeError(
+ "CAST(PARSE_JSON('[1, null, 3]') AS ARRAY<INT
NOT NULL>)",
+ "NOT NULL element type")
+ // a stored string is never parsed into an integer
+ .testTableApiRuntimeError(
+ call("PARSE_JSON", "[\"1\",
\"2\"]").cast(ARRAY(INT())),
+ "does not change the type")
+ .testSqlRuntimeError(
+ "CAST(PARSE_JSON('[\"1\", \"2\"]') AS
ARRAY<INT>)",
+ "does not change the type")
+ .testResult(
+ call("PARSE_JSON", "[\"1\",
\"2\"]").tryCast(ARRAY(INT())),
+ "TRY_CAST(PARSE_JSON('[\"1\", \"2\"]') AS
ARRAY<INT>)",
+ null,
+ ARRAY(INT()))
+ // a heterogeneous array fails on the first element
that is not an integer
+ .testTableApiRuntimeError(
+ call("PARSE_JSON", "[1, \"a\", 2,
\"b\"]").cast(ARRAY(INT())),
+ "does not change the type")
+ // a fractional element cannot narrow to INT without
dropping digits
+ .testTableApiRuntimeError(
+ call("PARSE_JSON", "[1.23, 2.45,
3.67]").cast(ARRAY(INT())),
+ "lose precision")
+ .testSqlRuntimeError(
+ "CAST(PARSE_JSON('[1.23, 2.45, 3.67]') AS
ARRAY<INT>)",
+ "lose precision")
+ // TRY_CAST swallows the failure and returns NULL for
the whole array
+ .testResult(
+ call("PARSE_JSON", "[1.23, 2.45,
3.67]").tryCast(ARRAY(INT())),
+ "TRY_CAST(PARSE_JSON('[1.23, 2.45, 3.67]') AS
ARRAY<INT>)",
+ null,
+ ARRAY(INT()))
+ // an object is not an array
+ .testTableApiRuntimeError(
+ call(
+ "PARSE_JSON",
+ "{\"id\": 7, \"name\":
\"ada\", \"active\": true}")
+ .cast(ARRAY(INT())),
+ "requires an array")
+ // ARRAY<VARIANT> shreds one level and keeps the
elements as variants, which
+ // then cast back to INT unchanged
+ .testResult(
+ call("PARSE_JSON", "[1, 2, 3]")
+ .cast(ARRAY(VARIANT()))
+ .cast(ARRAY(INT())),
+ "CAST(CAST(PARSE_JSON('[1, 2, 3]') AS
ARRAY<VARIANT>) AS ARRAY<INT>)",
+ new Integer[] {1, 2, 3},
+ ARRAY(INT()).notNull())
+ // the recursion composes for a nested array of arrays
+ .testResult(
+ call("PARSE_JSON", "[[1, 2],
[3]]").cast(ARRAY(ARRAY(INT()))),
+ "CAST(PARSE_JSON('[[1, 2], [3]]') AS
ARRAY<ARRAY<INT>>)",
+ new Integer[][] {{1, 2}, {3}},
+ ARRAY(ARRAY(INT())).notNull())
+ // a top-level VARIANT null casts to SQL NULL for a
nullable target
+ .testResult(
+ call("TRY_PARSE_JSON",
"null").cast(ARRAY(INT())),
+ "CAST(TRY_PARSE_JSON('null') AS ARRAY<INT>)",
+ null,
+ ARRAY(INT()))
+ // an element with no variant counterpart is rejected
at validation
+ .testTableApiValidationError(
+ call("PARSE_JSON",
"[1]").cast(ARRAY(INTERVAL(MONTH()))),
+ "Unsupported cast"));
+ }
+
private static List<TestSetSpec> allTypesBasic() {
return Arrays.asList(
CastTestSpecBuilder.testCastTo(BOOLEAN())
diff --git
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRuleProviderTest.java
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRuleProviderTest.java
index adc44c474be..fabe82749f4 100644
---
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRuleProviderTest.java
+++
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRuleProviderTest.java
@@ -27,6 +27,7 @@ import org.apache.flink.table.types.logical.VarCharType;
import org.junit.jupiter.api.Test;
+import static org.apache.flink.table.api.DataTypes.ARRAY;
import static org.apache.flink.table.api.DataTypes.BIGINT;
import static org.apache.flink.table.api.DataTypes.BOOLEAN;
import static org.apache.flink.table.api.DataTypes.BYTES;
@@ -34,6 +35,9 @@ import static org.apache.flink.table.api.DataTypes.DATE;
import static org.apache.flink.table.api.DataTypes.DECIMAL;
import static org.apache.flink.table.api.DataTypes.FIELD;
import static org.apache.flink.table.api.DataTypes.INT;
+import static org.apache.flink.table.api.DataTypes.INTERVAL;
+import static org.apache.flink.table.api.DataTypes.MONTH;
+import static org.apache.flink.table.api.DataTypes.MULTISET;
import static org.apache.flink.table.api.DataTypes.ROW;
import static org.apache.flink.table.api.DataTypes.STRING;
import static org.apache.flink.table.api.DataTypes.STRUCTURED;
@@ -136,4 +140,21 @@ class CastRuleProviderTest {
assertThat(CastRuleProvider.resolve(VARIANT, STRING_TYPE))
.isSameAs(VariantToStringCastRule.INSTANCE);
}
+
+ @Test
+ void testResolveVariantToArray() {
+ assertThat(CastRuleProvider.resolve(VARIANT,
ARRAY(INT()).getLogicalType()))
+ .isSameAs(VariantToArrayCastRule.INSTANCE);
+
+ // the element recurses through the VARIANT rules, including the
identity leaf and nesting
+ assertThat(CastRuleProvider.exists(VARIANT,
ARRAY(VARIANT()).getLogicalType())).isTrue();
+ assertThat(CastRuleProvider.exists(VARIANT,
ARRAY(ARRAY(INT())).getLogicalType())).isTrue();
+ assertThat(CastRuleProvider.canFail(VARIANT,
ARRAY(INT()).getLogicalType())).isTrue();
+
+ // an element with no variant counterpart makes the whole cast
unresolvable
+ assertThat(CastRuleProvider.exists(VARIANT,
ARRAY(INTERVAL(MONTH())).getLogicalType()))
+ .isFalse();
+ // MULTISET has no variant counterpart
+ assertThat(CastRuleProvider.exists(VARIANT,
MULTISET(STRING()).getLogicalType())).isFalse();
+ }
}
diff --git
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java
index e00c99c55ce..99d4b5aba14 100644
---
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java
+++
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java
@@ -42,6 +42,7 @@ import org.apache.flink.table.types.logical.StructuredType;
import org.apache.flink.table.utils.DateTimeUtils;
import org.apache.flink.types.bitmap.Bitmap;
import org.apache.flink.types.variant.Variant;
+import org.apache.flink.types.variant.VariantBuilder;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
@@ -167,27 +168,83 @@ class CastRulesTest {
/** U+1D54F, one code point but two UTF-16 units and four UTF-8 bytes. */
private static final String NON_BMP = "𝕏";
+ private static final VariantBuilder VARIANT_BUILDER = Variant.newBuilder();
private static final Variant VARIANT_ARRAY =
- Variant.newBuilder()
+ VARIANT_BUILDER
.array()
- .add(Variant.newBuilder().of(1))
- .add(Variant.newBuilder().of("two"))
- .add(Variant.newBuilder().of(false))
- .add(Variant.newBuilder().ofNull())
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.of("two"))
+ .add(VARIANT_BUILDER.of(false))
+ .add(VARIANT_BUILDER.ofNull())
.build();
private static final Variant VARIANT_OBJECT =
- Variant.newBuilder()
+ VARIANT_BUILDER
.object()
.add(
"k",
- Variant.newBuilder()
+ VARIANT_BUILDER
.array()
- .add(Variant.newBuilder().of(1))
- .add(Variant.newBuilder().of(2))
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.of(2))
.build())
.build();
+ /** {@code [1, 2, 3]}, the design's running array value. */
+ private static final Variant VARIANT_INT_ARRAY =
+ VARIANT_BUILDER
+ .array()
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.of(2))
+ .add(VARIANT_BUILDER.of(3))
+ .build();
+
+ /** {@code [1, null, 3]}, an array carrying a VARIANT null element. */
+ private static final Variant VARIANT_INT_ARRAY_WITH_NULL =
+ VARIANT_BUILDER
+ .array()
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.ofNull())
+ .add(VARIANT_BUILDER.of(3))
+ .build();
+
+ /**
+ * {@code ["1", "22", "333"]}, stored strings of different lengths a
numeric leaf must not
+ * parse.
+ */
+ private static final Variant VARIANT_STRING_ARRAY =
+ VARIANT_BUILDER
+ .array()
+ .add(VARIANT_BUILDER.of("1"))
+ .add(VARIANT_BUILDER.of("22"))
+ .add(VARIANT_BUILDER.of("333"))
+ .build();
+
+ /** {@code [1, "a", 2, "b"]}, a heterogeneous array of integers and
strings. */
+ private static final Variant VARIANT_MIXED_ARRAY =
+ VARIANT_BUILDER
+ .array()
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.of("a"))
+ .add(VARIANT_BUILDER.of(2))
+ .add(VARIANT_BUILDER.of("b"))
+ .build();
+
+ /** {@code [[1, 2], [3]]}, a nested array of arrays. */
+ private static final Variant VARIANT_NESTED_ARRAY =
+ VARIANT_BUILDER
+ .array()
+ .add(
+ VARIANT_BUILDER
+ .array()
+ .add(VARIANT_BUILDER.of(1))
+ .add(VARIANT_BUILDER.of(2))
+ .build())
+
.add(VARIANT_BUILDER.array().add(VARIANT_BUILDER.of(3)).build())
+ .build();
+
+ private static final Variant VARIANT_EMPTY_ARRAY =
VARIANT_BUILDER.array().build();
+
private static final DataType MY_STRUCTURED_TYPE =
STRUCTURED(
MyStructuredType.class,
@@ -1576,47 +1633,47 @@ class CastRulesTest {
// A character string renders like a regular cast of the
stored kind, so these
// expectations reuse the constants of the native cases above.
CastTestSpecBuilder.testCastTo(STRING())
- .fromCase(VARIANT(), Variant.newBuilder().of(true),
fromString("TRUE"))
- .fromCase(VARIANT(), Variant.newBuilder().of(false),
fromString("FALSE"))
- .fromCase(VARIANT(), Variant.newBuilder().of("foo"),
fromString("foo"))
- .fromCase(VARIANT(), Variant.newBuilder().of(42),
fromString("42"))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(true),
fromString("TRUE"))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(false),
fromString("FALSE"))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("foo"),
fromString("foo"))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42),
fromString("42"))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.456")),
+ VARIANT_BUILDER.of(new BigDecimal("123.456")),
fromString("123.456"))
// a small scale stays plain instead of turning into
scientific notation
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("0.0000000001")),
+ VARIANT_BUILDER.of(new
BigDecimal("0.0000000001")),
fromString("0.0000000001"))
.fromCase(
VARIANT(),
-
Variant.newBuilder().of(LocalDate.parse("2021-09-24")),
+
VARIANT_BUILDER.of(LocalDate.parse("2021-09-24")),
DATE_STRING)
.fromCase(
VARIANT(),
-
Variant.newBuilder().of(TIMESTAMP.toLocalDateTime()),
+
VARIANT_BUILDER.of(TIMESTAMP.toLocalDateTime()),
TIMESTAMP_STRING)
.fromCase(
VARIANT(),
CET_CONTEXT,
- Variant.newBuilder().of(TIMESTAMP.toInstant()),
+ VARIANT_BUILDER.of(TIMESTAMP.toInstant()),
TIMESTAMP_STRING_CET)
// a binary value is read as UTF-8, like a regular
BINARY to string cast
.fromCase(
VARIANT(),
-
Variant.newBuilder().of("hello".getBytes(StandardCharsets.UTF_8)),
+
VARIANT_BUILDER.of("hello".getBytes(StandardCharsets.UTF_8)),
fromString("hello"))
// a multi-byte sequence passes the well-formedness
check unchanged
.fromCase(
VARIANT(),
-
Variant.newBuilder().of("héllo".getBytes(StandardCharsets.UTF_8)),
+
VARIANT_BUILDER.of("héllo".getBytes(StandardCharsets.UTF_8)),
fromString("héllo"))
// bytes that are not valid UTF-8 are rejected rather
than decoded into
// the U+FFFD replacement character
.fail(
VARIANT(),
- Variant.newBuilder().of(INVALID_UTF8),
+ VARIANT_BUILDER.of(INVALID_UTF8),
TableRuntimeException.class)
// an object or an array has no scalar rendering and
fails the cast
.fail(VARIANT(), VARIANT_ARRAY,
TableRuntimeException.class)
@@ -1627,218 +1684,192 @@ class CastRulesTest {
VARIANT(), VARIANT_ARRAY,
fromString("[1,\"two\",false,null]"))
.fromCasePrinting(VARIANT(), VARIANT_OBJECT,
fromString("{\"k\":[1,2]}"))
.fromCasePrinting(
- VARIANT(), Variant.newBuilder().of("foo"),
fromString("\"foo\""))
- .fromCasePrinting(VARIANT(),
Variant.newBuilder().of(42), fromString("42")),
+ VARIANT(), VARIANT_BUILDER.of("foo"),
fromString("\"foo\""))
+ .fromCasePrinting(VARIANT(), VARIANT_BUILDER.of(42),
fromString("42")),
// A bounded character target pads and trims like any other
cast into it, and its
// length counts code points, so a character outside the BMP
fills one position
// even though it occupies two UTF-16 units.
CastTestSpecBuilder.testCastTo(CHAR(1))
- .fromCase(VARIANT(), Variant.newBuilder().of("x"),
fromString("x"))
- .fromCase(VARIANT(), Variant.newBuilder().of(NON_BMP),
fromString(NON_BMP))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("x"),
fromString("x"))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(NON_BMP),
fromString(NON_BMP))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(NON_BMP + NON_BMP),
+ VARIANT_BUILDER.of(NON_BMP + NON_BMP),
fromString(NON_BMP))
- .fromCase(
- VARIANT(),
Variant.newBuilder().of("abcdefghij"), fromString("a")),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("abcdefghij"),
fromString("a")),
CastTestSpecBuilder.testCastTo(CHAR(5))
// shorter than the target, so it is padded to the
fixed width
- .fromCase(VARIANT(), Variant.newBuilder().of("ab"),
fromString("ab "))
- .fromCase(
- VARIANT(),
- Variant.newBuilder().of("abcdefghij"),
- fromString("abcde")),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("ab"),
fromString("ab "))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("abcdefghij"),
fromString("abcde")),
CastTestSpecBuilder.testCastTo(VARCHAR(2))
- .fromCase(VARIANT(), Variant.newBuilder().of(NON_BMP),
fromString(NON_BMP))
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(NON_BMP),
fromString(NON_BMP))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(NON_BMP + NON_BMP),
+ VARIANT_BUILDER.of(NON_BMP + NON_BMP),
fromString(NON_BMP + NON_BMP))
// longer than the target, so it is trimmed rather
than rejected, and a
// variable width target is not padded
.fromCase(
VARIANT(),
- Variant.newBuilder().of(NON_BMP + NON_BMP +
NON_BMP),
+ VARIANT_BUILDER.of(NON_BMP + NON_BMP +
NON_BMP),
fromString(NON_BMP + NON_BMP))
- .fromCase(VARIANT(), Variant.newBuilder().of("a"),
fromString("a")),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of("a"),
fromString("a")),
CastTestSpecBuilder.testCastTo(BOOLEAN())
- .fromCase(VARIANT(), Variant.newBuilder().of(true),
true)
- .fromCase(VARIANT(), Variant.newBuilder().of(false),
false)
- .fail(VARIANT(), Variant.newBuilder().of(1),
TableRuntimeException.class),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(true), true)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(false), false)
+ .fail(VARIANT(), VARIANT_BUILDER.of(1),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(TINYINT())
- .fromCase(VARIANT(), Variant.newBuilder().of((byte)
42), (byte) 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of((byte) 42),
(byte) 42)
// a wider integer kind narrows while the value is in
range
- .fromCase(VARIANT(), Variant.newBuilder().of(42),
(byte) 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42), (byte) 42)
// out of range is rejected instead of wrapping
- .fail(VARIANT(), Variant.newBuilder().of(1000),
TableRuntimeException.class)
- .fail(VARIANT(), Variant.newBuilder().of("x"),
TableRuntimeException.class),
+ .fail(VARIANT(), VARIANT_BUILDER.of(1000),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of("x"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(SMALLINT())
- .fromCase(VARIANT(), Variant.newBuilder().of((short)
42), (short) 42)
- .fromCase(VARIANT(), Variant.newBuilder().of((byte)
42), (short) 42)
- .fromCase(VARIANT(), Variant.newBuilder().of(1000),
(short) 1000)
- .fail(
- VARIANT(),
- Variant.newBuilder().of(40000),
- TableRuntimeException.class)
- .fail(
- VARIANT(),
- Variant.newBuilder().of(true),
- TableRuntimeException.class),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of((short) 42),
(short) 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of((byte) 42),
(short) 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(1000), (short)
1000)
+ .fail(VARIANT(), VARIANT_BUILDER.of(40000),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(true),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(INT())
- .fromCase(VARIANT(), Variant.newBuilder().of(42), 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42), 42)
// every integer kind converts as long as the value
fits
- .fromCase(VARIANT(), Variant.newBuilder().of((byte)
42), 42)
- .fromCase(VARIANT(), Variant.newBuilder().of((short)
42), 42)
- .fromCase(VARIANT(), Variant.newBuilder().of(42L), 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of((byte) 42), 42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of((short) 42),
42)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42L), 42)
.fail(
VARIANT(),
- Variant.newBuilder().of(2147483648L),
+ VARIANT_BUILDER.of(2147483648L),
TableRuntimeException.class)
// an approximate or decimal kind converts when the
value is integral
- .fromCase(VARIANT(), Variant.newBuilder().of(7.0d), 7)
- .fromCase(VARIANT(), Variant.newBuilder().of(new
BigDecimal("7.0")), 7)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(7.0d), 7)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(new
BigDecimal("7.0")), 7)
// a fractional value would have to be rounded away,
so it is rejected
- .fail(VARIANT(), Variant.newBuilder().of(7.2d),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(7.2d),
TableRuntimeException.class)
.fail(
VARIANT(),
- Variant.newBuilder().of(new BigDecimal("7.2")),
+ VARIANT_BUILDER.of(new BigDecimal("7.2")),
TableRuntimeException.class)
// a non-numeric variant cannot be cast to a number
- .fail(
- VARIANT(),
- Variant.newBuilder().of("foo"),
- TableRuntimeException.class)
- .fail(
- VARIANT(),
- Variant.newBuilder().of(true),
- TableRuntimeException.class),
+ .fail(VARIANT(), VARIANT_BUILDER.of("foo"),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(true),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(BIGINT())
- .fromCase(VARIANT(), Variant.newBuilder().of(42L), 42L)
- .fromCase(VARIANT(), Variant.newBuilder().of(42), 42L)
- .fail(VARIANT(), Variant.newBuilder().of("x"),
TableRuntimeException.class),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42L), 42L)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(42), 42L)
+ .fail(VARIANT(), VARIANT_BUILDER.of("x"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(FLOAT())
// every numeric kind reaches an approximate target
- .fromCase(VARIANT(), Variant.newBuilder().of(1.5f),
1.5f)
- .fromCase(VARIANT(), Variant.newBuilder().of(1.5d),
1.5f)
- .fromCase(VARIANT(), Variant.newBuilder().of(3), 3.0f)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(1.5f), 1.5f)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(1.5d), 1.5f)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(3), 3.0f)
.fromCase(
- VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.456")),
- 123.456f)
+ VARIANT(), VARIANT_BUILDER.of(new
BigDecimal("123.456")), 123.456f)
// a magnitude a FLOAT cannot represent is still
rejected
- .fail(
- VARIANT(),
- Variant.newBuilder().of(1e40d),
- TableRuntimeException.class)
- .fail(VARIANT(), Variant.newBuilder().of("x"),
TableRuntimeException.class),
+ .fail(VARIANT(), VARIANT_BUILDER.of(1e40d),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of("x"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(DOUBLE())
- .fromCase(VARIANT(), Variant.newBuilder().of(1.5d),
1.5d)
- .fromCase(VARIANT(), Variant.newBuilder().of(1.5f),
1.5d)
- .fromCase(VARIANT(), Variant.newBuilder().of(3), 3.0d)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(1.5d), 1.5d)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(1.5f), 1.5d)
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(3), 3.0d)
.fromCase(
- VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.456")),
- 123.456d)
- .fail(VARIANT(), Variant.newBuilder().of("x"),
TableRuntimeException.class),
+ VARIANT(), VARIANT_BUILDER.of(new
BigDecimal("123.456")), 123.456d)
+ .fail(VARIANT(), VARIANT_BUILDER.of("x"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(DECIMAL(5, 2))
.fromCase(VARIANT(), null, null)
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.45")),
+ VARIANT_BUILDER.of(new BigDecimal("123.45")),
DecimalData.fromBigDecimal(new
BigDecimal("123.45"), 5, 2))
// trailing zeros may be appended to reach the target
scale
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.4")),
+ VARIANT_BUILDER.of(new BigDecimal("123.4")),
DecimalData.fromBigDecimal(new
BigDecimal("123.40"), 5, 2))
// an integer is exact, so it converts when it fits
.fromCase(
VARIANT(),
- Variant.newBuilder().of(42),
+ VARIANT_BUILDER.of(42),
DecimalData.fromBigDecimal(new
BigDecimal("42.00"), 5, 2))
// a scale that would have to round is rejected
.fail(
VARIANT(),
- Variant.newBuilder().of(new
BigDecimal("123.456")),
+ VARIANT_BUILDER.of(new BigDecimal("123.456")),
TableRuntimeException.class)
// an approximate kind is not read as a decimal
- .fail(VARIANT(), Variant.newBuilder().of(1.5d),
TableRuntimeException.class)
- .fail(VARIANT(), Variant.newBuilder().of("x"),
TableRuntimeException.class),
+ .fail(VARIANT(), VARIANT_BUILDER.of(1.5d),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of("x"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(BYTES())
.fromCase(VARIANT(), null, null)
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2, 3}),
+ VARIANT_BUILDER.of(new byte[] {1, 2, 3}),
new byte[] {1, 2, 3})
// the raw bytes stay reachable when the character
string cast rejects
// them, which is what makes this the way to inspect
such a value
- .fromCase(VARIANT(),
Variant.newBuilder().of(INVALID_UTF8), INVALID_UTF8)
- .fail(
- VARIANT(),
- Variant.newBuilder().of("foo"),
- TableRuntimeException.class),
+ .fromCase(VARIANT(), VARIANT_BUILDER.of(INVALID_UTF8),
INVALID_UTF8)
+ .fail(VARIANT(), VARIANT_BUILDER.of("foo"),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(DATE())
.fromCase(
VARIANT(),
- Variant.newBuilder().of(LocalDate.of(2020, 1,
1)),
+ VARIANT_BUILDER.of(LocalDate.of(2020, 1, 1)),
(int) LocalDate.of(2020, 1, 1).toEpochDay())
- .fail(VARIANT(), Variant.newBuilder().of(1),
TableRuntimeException.class),
+ .fail(VARIANT(), VARIANT_BUILDER.of(1),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(TIMESTAMP())
.fromCase(VARIANT(), null, null)
.fromCase(
VARIANT(),
- Variant.newBuilder().of(LocalDateTime.of(2020,
1, 1, 12, 0, 0)),
+ VARIANT_BUILDER.of(LocalDateTime.of(2020, 1,
1, 12, 0, 0)),
TimestampData.fromLocalDateTime(
LocalDateTime.of(2020, 1, 1, 12, 0,
0)))
- .fail(VARIANT(), Variant.newBuilder().of(1),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(1),
TableRuntimeException.class)
// a TIMESTAMP_LTZ is a different kind and is not read
as a TIMESTAMP
.fail(
VARIANT(),
-
Variant.newBuilder().of(Instant.ofEpochSecond(1_600_000_000L)),
+
VARIANT_BUILDER.of(Instant.ofEpochSecond(1_600_000_000L)),
TableRuntimeException.class),
// A variant keeps microseconds, so fractional seconds beyond
the target precision
// are truncated, matching a regular cast into a narrower
TIMESTAMP.
CastTestSpecBuilder.testCastTo(TIMESTAMP(3))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(LocalDateTime.of(2020,
1, 1, 12, 0, 0)),
+ VARIANT_BUILDER.of(LocalDateTime.of(2020, 1,
1, 12, 0, 0)),
TimestampData.fromLocalDateTime(
LocalDateTime.of(2020, 1, 1, 12, 0,
0)))
.fromCase(
VARIANT(),
- Variant.newBuilder()
- .of(LocalDateTime.of(2020, 1, 1, 12,
0, 0, 123000000)),
+ VARIANT_BUILDER.of(
+ LocalDateTime.of(2020, 1, 1, 12, 0, 0,
123000000)),
TimestampData.fromLocalDateTime(
LocalDateTime.of(2020, 1, 1, 12, 0, 0,
123000000)))
.fromCase(
VARIANT(),
- Variant.newBuilder()
- .of(LocalDateTime.of(2020, 1, 1, 12,
0, 0, 123456000)),
+ VARIANT_BUILDER.of(
+ LocalDateTime.of(2020, 1, 1, 12, 0, 0,
123456000)),
TimestampData.fromLocalDateTime(
LocalDateTime.of(2020, 1, 1, 12, 0, 0,
123000000))),
CastTestSpecBuilder.testCastTo(TIMESTAMP(0))
.fromCase(
VARIANT(),
- Variant.newBuilder()
- .of(LocalDateTime.of(2020, 1, 1, 12,
0, 0, 123000000)),
+ VARIANT_BUILDER.of(
+ LocalDateTime.of(2020, 1, 1, 12, 0, 0,
123000000)),
TimestampData.fromLocalDateTime(
LocalDateTime.of(2020, 1, 1, 12, 0,
0))),
CastTestSpecBuilder.testCastTo(TIMESTAMP_LTZ())
.fromCase(
VARIANT(),
-
Variant.newBuilder().of(Instant.ofEpochSecond(1_600_000_000L)),
+
VARIANT_BUILDER.of(Instant.ofEpochSecond(1_600_000_000L)),
TimestampData.fromInstant(Instant.ofEpochSecond(1_600_000_000L)))
- .fail(VARIANT(), Variant.newBuilder().of(1),
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(1),
TableRuntimeException.class)
// a TIMESTAMP is not read as a TIMESTAMP_LTZ either
.fail(
VARIANT(),
- Variant.newBuilder().of(LocalDateTime.of(2020,
1, 1, 12, 0, 0)),
+ VARIANT_BUILDER.of(LocalDateTime.of(2020, 1,
1, 12, 0, 0)),
TableRuntimeException.class),
CastTestSpecBuilder.testCastTo(TIMESTAMP_LTZ(3))
.fromCase(
VARIANT(),
- Variant.newBuilder()
-
.of(Instant.ofEpochSecond(1_600_000_000L, 123456000)),
+ VARIANT_BUILDER.of(
+ Instant.ofEpochSecond(1_600_000_000L,
123456000)),
TimestampData.fromInstant(
Instant.ofEpochSecond(1_600_000_000L,
123000000))),
// A binary target pads a shorter value and truncates a longer
one, matching a
@@ -1846,26 +1877,121 @@ class CastRulesTest {
CastTestSpecBuilder.testCastTo(BINARY(4))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2, 3,
4}),
+ VARIANT_BUILDER.of(new byte[] {1, 2, 3, 4}),
new byte[] {1, 2, 3, 4})
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2}),
+ VARIANT_BUILDER.of(new byte[] {1, 2}),
new byte[] {1, 2, 0, 0})
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2, 3,
4, 5, 6}),
+ VARIANT_BUILDER.of(new byte[] {1, 2, 3, 4, 5,
6}),
new byte[] {1, 2, 3, 4}),
CastTestSpecBuilder.testCastTo(VARBINARY(4))
// a variable width target is trimmed but never padded
+ .fromCase(
+ VARIANT(), VARIANT_BUILDER.of(new byte[] {1,
2}), new byte[] {1, 2})
+ .fromCase(
+ VARIANT(),
+ VARIANT_BUILDER.of(new byte[] {1, 2, 3, 4, 5,
6}),
+ new byte[] {1, 2, 3, 4}),
+ // From VARIANT to a constructed target. A constructed cast is
the scalar cast
+ // applied to every leaf plus a shape check at each level.
+ CastTestSpecBuilder.testCastTo(ARRAY(INT()))
+ .fromCase(VARIANT(), null, null)
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY,
+ new GenericArrayData(new Integer[] {1, 2, 3}))
+ // a VARIANT null element maps to SQL NULL for a
nullable element type
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY_WITH_NULL,
+ new GenericArrayData(new Integer[] {1, null,
3}))
+ .fromCase(
+ VARIANT(),
+ VARIANT_EMPTY_ARRAY,
+ new GenericArrayData(new Integer[] {}))
+ // a stored string is never parsed into an integer
+ .fail(VARIANT(), VARIANT_STRING_ARRAY,
TableRuntimeException.class)
+ // a heterogeneous array fails on an element that is
not an integer
+ .fail(VARIANT(), VARIANT_MIXED_ARRAY,
TableRuntimeException.class)
+ // an object or a scalar is not an array
+ .fail(VARIANT(), VARIANT_OBJECT,
TableRuntimeException.class)
+ .fail(VARIANT(), VARIANT_BUILDER.of(1),
TableRuntimeException.class),
+ CastTestSpecBuilder.testCastTo(ARRAY(INT().notNull()))
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY,
+ new GenericArrayData(new int[] {1, 2, 3}))
+ // a VARIANT null element fails a NOT NULL element type
+ .fail(VARIANT(), VARIANT_INT_ARRAY_WITH_NULL,
TableRuntimeException.class),
+ CastTestSpecBuilder.testCastTo(ARRAY(STRING()))
+ // each element renders to string like the scalar cast
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY,
+ new GenericArrayData(
+ new Object[] {
+ fromString("1"), fromString("2"),
fromString("3")
+ }))
+ // a heterogeneous array renders every element to
string
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2}),
- new byte[] {1, 2})
+ VARIANT_MIXED_ARRAY,
+ new GenericArrayData(
+ new Object[] {
+ fromString("1"),
+ fromString("a"),
+ fromString("2"),
+ fromString("b")
+ }))
+ // stored strings of different lengths render unchanged
+ .fromCase(
+ VARIANT(),
+ VARIANT_STRING_ARRAY,
+ new GenericArrayData(
+ new Object[] {
+ fromString("1"), fromString("22"),
fromString("333")
+ })),
+ CastTestSpecBuilder.testCastTo(ARRAY(DOUBLE()))
+ // an approximate leaf takes any numeric kind
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY,
+ new GenericArrayData(new Double[] {1.0, 2.0,
3.0})),
+ // the recursion composes: an array of arrays with no special
case
+ CastTestSpecBuilder.testCastTo(ARRAY(ARRAY(INT())))
+ .fromCase(
+ VARIANT(),
+ VARIANT_NESTED_ARRAY,
+ new GenericArrayData(
+ new GenericArrayData[] {
+ new GenericArrayData(new Integer[]
{1, 2}),
+ new GenericArrayData(new Integer[]
{3})
+ })),
+ // an ARRAY<VARIANT> leaf is the identity cast, keeping each
element as a variant
+ CastTestSpecBuilder.testCastTo(ARRAY(VARIANT()))
.fromCase(
VARIANT(),
- Variant.newBuilder().of(new byte[] {1, 2, 3,
4, 5, 6}),
- new byte[] {1, 2, 3, 4}));
+ VARIANT_INT_ARRAY,
+ new GenericArrayData(
+ new Variant[] {
+ VARIANT_INT_ARRAY.getElement(0),
+ VARIANT_INT_ARRAY.getElement(1),
+ VARIANT_INT_ARRAY.getElement(2)
+ }))
+ // the identity cast keeps a VARIANT null element as a
variant null, not a
+ // SQL NULL
+ .fromCase(
+ VARIANT(),
+ VARIANT_INT_ARRAY_WITH_NULL,
+ new GenericArrayData(
+ new Variant[] {
+
VARIANT_INT_ARRAY_WITH_NULL.getElement(0),
+
VARIANT_INT_ARRAY_WITH_NULL.getElement(1),
+
VARIANT_INT_ARRAY_WITH_NULL.getElement(2)
+ })));
}
@TestFactory
diff --git
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/VariantCastUtils.java
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/VariantCastUtils.java
index 89f112348fe..ac8d550195c 100644
---
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/VariantCastUtils.java
+++
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/VariantCastUtils.java
@@ -63,6 +63,27 @@ public final class VariantCastUtils {
private VariantCastUtils() {}
+ /**
+ * Reads the size of an array variant, failing when the variant is not an
array. A constructed
+ * cast checks the shape at every level: only an array casts to {@code
ARRAY}.
+ */
+ public static int arraySize(Variant variant, String targetType) {
+ if (variant.isArray()) {
+ return variant.getArraySize();
+ }
+ throw wrongShape(variant, targetType, "an array");
+ }
+
+ private static TableRuntimeException wrongShape(
+ Variant variant, String targetType, String required) {
+ return new TableRuntimeException(
+ String.format(
+ "Cannot cast a VARIANT %s value to %s because the
target requires %s. Only "
+ + "a variant array casts to ARRAY and only a
variant object casts "
+ + "to ROW, STRUCTURED, or MAP.",
+ variant.getType(), targetType, required));
+ }
+
/**
* Reads a numeric variant as a {@code long} and checks it against the
target range. An
* approximate or decimal value is accepted only when it is already
integral, so nothing is
