[ https://issues.apache.org/jira/browse/HIVE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang updated HIVE-8810: ------------------------------ Fix Version/s: (was: spark-branch) 1.1.0 > Make HashTableSinkOperator works for Spark Branch [Spark Branch] > ---------------------------------------------------------------- > > Key: HIVE-8810 > URL: https://issues.apache.org/jira/browse/HIVE-8810 > Project: Hive > Issue Type: Sub-task > Components: Spark > Affects Versions: spark-branch > Reporter: Chao Sun > Assignee: Jimmy Xiang > Fix For: 1.1.0 > > Attachments: HIVE-8810.1-spark.patch, HIVE-8810.2-spark.patch, > HIVE-8810.3-spark.patch > > > In MR, all small tables for a particular MJ operator share the same instance > of {{HashTableSinkOperator}}, while in Spark branch, each small table > corresponds to a different {{HashTableSinkOperator}} instance. This > difference causes some issues. > For instance, in {{HashTableSinkOperator#processOp}}, it uses a tag to look > for information in various data structures, such as {{joinKeys}}, > {{filterMaps}}, {{joinValues}}, etc. Those data structures stores the > information BEFORE it splits the MJ operator with its parents. But, since > later on we use separate {{HashTableSinkOperator}} for each small table, that > information is no longer valid, and thus this method will fail. > This JIRA is to track and solve these related issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)