[
https://issues.apache.org/jira/browse/HIVE-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198723#comment-14198723
]
Suhas Satish commented on HIVE-8622:
------------------------------------
I saw this condition in your patch -
if (containsOp(work, MapJoinOperator.class)) {
if (containsOp(parentWork, HashTableSinkOperator.class)) {
This means that HIVE-8621 which introduces
*replaceReduceSinkWithHashTableSink(..)* should be called before this stage.
To create HashTableSinkOperator, we need to pass in the MapJoinOperator
associated with it. This is available in *GenSparkProcContext* but that doesnt
get passed into the physical resolvers. We have to either pass it in or find
another way to extract this information from the available physicalContext
inside *SparkMapJoinResolver* and pass it into
*replaceReduceSinkWithHashTableSink(..)*
> Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]
> ----------------------------------------------------------------
>
> Key: HIVE-8622
> URL: https://issues.apache.org/jira/browse/HIVE-8622
> Project: Hive
> Issue Type: Sub-task
> Reporter: Suhas Satish
> Assignee: Chao
> Attachments: HIVE-8622.patch
>
>
> This is a sub-task of map-join for spark
> https://issues.apache.org/jira/browse/HIVE-7613
> This can use the baseline patch for map-join
> https://issues.apache.org/jira/browse/HIVE-8616
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
