[
https://issues.apache.org/jira/browse/FLINK-28343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561331#comment-17561331
]
luoyuxia commented on FLINK-28343:
----------------------------------
[~tartarus] Thanks for reporting. Should be fixed after
https://issues.apache.org/jira/browse/FLINK-25645
> Hive dialect fails using union map type
> ---------------------------------------
>
> Key: FLINK-28343
> URL: https://issues.apache.org/jira/browse/FLINK-28343
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive
> Reporter: tartarus
> Priority: Major
>
> We can reproduce it with the following example
> {code:java}
> @Test
> public void testUnionMapType() {
> // automatically load hive module in hive-compatible mode
> HiveModule hiveModule = new HiveModule(hiveCatalog.getHiveVersion());
> CoreModule coreModule = CoreModule.INSTANCE;
> for (String loaded : tableEnv.listModules()) {
> tableEnv.unloadModule(loaded);
> }
> tableEnv.loadModule("hive", hiveModule);
> tableEnv.loadModule("core", coreModule);
> tableEnv.executeSql(
> "CREATE TABLE test_map_table (params string) PARTITIONED BY
> (`p_date` string)");
> tableEnv.executeSql("select map(\"\",\"\") as params from test_map_table
> union select map(\"\",\"\") as params from test_map_table");
> } {code}
> Because union semantics need to be de-duplicated, So flink will introduce an
> Aggregate,
> An exception will be thrown
> {code:java}
> Unsupported type(MAP) to generate hash code, the type(MAP) is not supported
> as a GROUP_BY/PARTITION_BY/JOIN_EQUAL/UNION field {code}
> We can see the Aggregate operator in the execution plan
> {code:java}
> optimize subquery_rewrite cost 33 ms.
> optimize result:
> LogicalSink(table=[*anonymous_collect$1*], fields=[params])
> +- LogicalProject(inputs=[0])
> +- LogicalAggregate(group=[{0}])
> +- LogicalProject(inputs=[0])
> +- LogicalUnion(all=[true])
> :- LogicalProject(exprs=[[map(_UTF-16LE'':VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE", _UTF-16LE'':VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE")]])
> : +- LogicalTableScan(table=[[test-catalog, default,
> test_map_table]])
> +- LogicalProject(exprs=[[map(_UTF-16LE'':VARCHAR(2147483647)
> CHARACTER SET "UTF-16LE", _UTF-16LE'':VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE")]])
> +- LogicalTableScan(table=[[test-catalog, default,
> test_map_table]]) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
