[
https://issues.apache.org/jira/browse/CALCITE-7812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120151#comment-18120151
]
Lino Rosa commented on CALCITE-7812:
------------------------------------
I think maybe an explicit example will help more:
{code:java}
// Added to TCatalogReader.java
final MockTable t4 =
MockTable.create(this, tSchema, "T4", false, 7.0, null);
t4.addColumn("t4_row",
typeFactory.builder()
.add("EXPR$0", f.doubleType) //named 'EXPR$0' to prevent name
difference to trigger a CAST during the test below
.build());
registerTable(t4);
{code}
Then you can see the cast is inserted:
{code:java}
// Added to TypeCoercionTest.java
@Test void testQuerySourceCoercionCastsRowOnFieldTypeAlone() {
final String sql = "insert into t4 select row(sum(t2_bigint)) from t2";
final SqlValidatorFixture f = sql(sql);
f.tester.validateAndThen(f.factory, f.sap, (sap, validator, node) -> {
final String expected =
"INSERT INTO `T4`\n"
+ "SELECT CAST(ROW(SUM(`T2_BIGINT`)) AS ROW(`EXPR$0` DOUBLE))\n"
+ "FROM `T2`";
assertThat(node.toSqlString(AnsiSqlDialect.DEFAULT).getSql(), is(expected));
});
} {code}
The decision to cast is made on {{{}AbstractTypeCoercion#needToCast{}}},
naturally.
It delegates to SqlTypeUtil#equalSansNullability, comparing equality of
{{RecordType(BIGINT EXPR$0)}} and {{{}RecordType(DOUBLE EXPR$0){}}}.
---
That said, I don't think scrutinizing the reasons of why the casts are injected
is the main point for this ticket specifically. The issue is really that
whenever they're added they end up enforcing nullability, which may break on a
target dialect whose nullability disagrees with Calcite's.
I hope the example above helps [~mbudiu]. Thanks for sticking around this long
thread.
> Implicit type coercion emits CASTs in unparsed SQL that the target dialect
> rejects
> ----------------------------------------------------------------------------------
>
> Key: CALCITE-7812
> URL: https://issues.apache.org/jira/browse/CALCITE-7812
> Project: Calcite
> Issue Type: Bug
> Reporter: Lino Rosa
> Assignee: krooswu
> Priority: Major
>
> h2. Problem
> When type coercion is on, {{TypeCoercionImpl}} rewrites the validated
> {{SqlNode}} tree. It wraps operands and select items in {{{}CAST(x AS T){}}},
> where {{T}} is the type Calcite's own rules chose. These casts become
> ordinary {{RexCall(CAST)}} nodes after {{{}SqlToRelConverter{}}}.
> {{RelToSqlConverter}} then unparses them into the SQL sent to the target
> engine.
> That is fine only when Calcite and the engine type the expression in a
> compatible way. Otherwise it generates invalid queries.
> h4. Example: struct fields and {{SUM}} nullability on Spark
> {code:java}
> -- target: t2 (s ROW(a DOUBLE))
> -- source: t1 (k INT NOT NULL, x BIGINT NOT NULL)
> INSERT INTO t2 (s)
> SELECT ROW(SUM(x)) FROM t1 GROUP BY k {code}
> # Calcite types {{SUM}} as {{{}BIGINT NOT NULL{}}}, because {{x}} is {{NOT
> NULL}} and the query is grouped.
> # {{coerceColumnType}} wraps the row in a {{CAST}} to the target struct type.
> # {{SqlTypeUtil.convertTypeToSpec}} loses per-field nullability
> (CALCITE-6932). The unparsed cast spells the field {{{}NOT NULL{}}}.
> # Spark types {{SUM}} as nullable. It refuses to cast a struct with a
> nullable field to one with a {{NOT NULL}} field, so the query fails at
> analysis. Fixing CALCITE-6932 alone would not settle this. The cast still
> carries Calcite's view of nullability, which is correct for Calcite and wrong
> for Spark. The engine would have accepted the uncast expression.
> h4. Existing workaround is not enough
> {{SqlDialect.supportsImplicitTypeCoercion}} together with
> {{SqlImplementor.stripCastFromString}} already recognizes that engines coerce
> for themselves. Its scope is limited but its existing is promising and may
> lead to a fix.
> h3. Proposals
> h4. 1. Carry the coerced type outside the tree
> h4. The validator would record the coerced type without rewriting the tree,
> and conversion would pick it up from there. This is the cleanest option but
> hard today, because the tree is the only place coercion is stored. Much of
> the downstream logic (e.g. {{FamilyOperandTypeChecker, }}{{SqlToRelConverter)
> rely on the existence of this CAST.}}
> h4. 2. Mark coercion casts, then let the dialect decide (preferred)
> h4. Keep inserting a cast, but make it distinguishable from a user cast. Two
> ways to do that:
> * A dedicated operator of kind {{{}CAST{}}}, for example
> {{{}SqlStdOperatorTable.IMPLICIT_CAST{}}}.
> * A flag on the cast.
> The marker has to survive {{{{{}SqlToRelConverter{}}}}} and appear on the
> {{{}RexCall{}}}. Because its kind stays {{{}CAST{}}}, {{RexSimplify}} and the
> rules keep treating it as a cast.
> And here we go back to {{SqlDialect.supportsImplicitTypeCoercion.}} We can
> now simply unparse this marker as either a \{{CAST }}or not unparse anything
> depending on that flag.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
