[
https://issues.apache.org/jira/browse/HIVE-28362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863199#comment-17863199
]
Shohei Okumiya commented on HIVE-28362:
---------------------------------------
Thanks. I understand VOID is troublesome not at the level of Hive tables but at
the level of a table format. Actually, I confirmed that the test case would
fail when I added the following configuration.
{code:java}
set hive.default.fileformat=ORC; {code}
{code:java}
See ./ql/target/tmp/log/hive.log or ./itests/qtest/target/tmp/log/hive.log, or
check ./ql/target/surefire-reports or ./itests/qtest/target/surefire-reports/
for specific test cases logs.
java.lang.IllegalArgumentException: Unknown primitive type VOID
at
org.apache.hadoop.hive.ql.io.orc.OrcStruct.createObjectInspector(OrcStruct.java:549)
at
org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcStructInspector.<init>(OrcStruct.java:197)
at
org.apache.hadoop.hive.ql.io.orc.OrcStruct.createObjectInspector(OrcStruct.java:552)
{code}
I definitely agree that the current PR is not a good approach. I closed it.
In my mind, we have three options. The first one is we support it in CTAS when
VOID in CTAS/CTE is legal. The second one is to disable materialization when
VOID in CTE is legal and it is reasonably implementable in Hive. The third one
is just to void this ticket and ask users to explicitly cast them. Let me check
SQL:2023 and the feasibility. I will likely choose the third one in my current
impression, though.
> Fail to materialize a CTE with VOID
> -----------------------------------
>
> Key: HIVE-28362
> URL: https://issues.apache.org/jira/browse/HIVE-28362
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Affects Versions: 4.0.0
> Reporter: Shohei Okumiya
> Assignee: Shohei Okumiya
> Priority: Major
> Labels: pull-request-available
>
> CTE materialization fails when it includes a NULL literal.
> {code:java}
> set hive.optimize.cte.materialize.full.aggregate.only=false;
> set hive.optimize.cte.materialize.threshold=2;
> WITH x AS (SELECT null AS null_value)
> SELECT * FROM x UNION ALL SELECT * FROM x; {code}
> Error message.
> {code:java}
> org.apache.hadoop.hive.ql.parse.SemanticException: CREATE-TABLE-AS-SELECT
> creates a VOID type, please use CAST to specify the type, near field:
> null_value
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.deriveFileSinkColTypes(SemanticAnalyzer.java:8344)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.deriveFileSinkColTypes(SemanticAnalyzer.java:8303)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(SemanticAnalyzer.java:7846)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11598)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:11461)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12397)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12263)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:638)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13136)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:465)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.materializeCTE(CalcitePlanner.java:1062)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2390)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2338)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2340)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2501)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2323)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:12978)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13085)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:465)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:332)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:109)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:508) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
