[
https://issues.apache.org/jira/browse/HIVE-8024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126398#comment-14126398
]
Chao commented on HIVE-8024:
----------------------------
I think there's a problem removing union op in place. Suppose there's a plan
for multi-insertion looks like this:
{code}
TS_0 TS_2
\ /
UNION_3
/ \
SEL_1 SEL_4
{code}
(I ignored some operators)
Currently, {{TS_0}} and {{TS_2}} will be in two MapWorks, which have separate
plans, like following:
{code}
TS_0 TS_2
/ \ / \
SEL_1 SEL_4 SEL_1 SEL_4
{code}
If we remove the union operator from the original tree, the result may not be
correct.
> Find out whether it's possible to remove UnionOperator from original operator
> tree [Spark Branch]
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-8024
> URL: https://issues.apache.org/jira/browse/HIVE-8024
> Project: Hive
> Issue Type: Task
> Components: Spark
> Reporter: Chao
> Assignee: Chao
>
> Currently, after operator tree is processed, the generated works with union
> operators will go through {{GenSparkUtils::removeUnionOperators}}, which will
> clone the original operator plan associated with the work, and remove union
> operators in it. This caused some issues as seen, for example, in HIVE-7870.
> This JIRA is created to find out whether it's possible to just remove the
> union operators in the original plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
