[jira] [Assigned] (SPARK-22671) SortMergeJoin read more data when wholeStageCodegen is off compared with when it is on
[ https://issues.apache.org/jira/browse/SPARK-22671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22671: Assignee: (was: Apache Spark) > SortMergeJoin read more data when wholeStageCodegen is off compared with when > it is on > -- > > Key: SPARK-22671 > URL: https://issues.apache.org/jira/browse/SPARK-22671 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Chenzhao Guo > > In SortMergeJoin(with wholeStageCodegen), an optimization already exists: if > the left table of a partition is empty then there is no need to read the > right table of this corresponding partition. This benefits the case in which > many partitions of left table is empty and the right table is big. > While in the code path without wholeStageCodegen, this optimization doesn't > happen. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-22671) SortMergeJoin read more data when wholeStageCodegen is off compared with when it is on
[ https://issues.apache.org/jira/browse/SPARK-22671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22671: Assignee: Apache Spark > SortMergeJoin read more data when wholeStageCodegen is off compared with when > it is on > -- > > Key: SPARK-22671 > URL: https://issues.apache.org/jira/browse/SPARK-22671 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Chenzhao Guo >Assignee: Apache Spark > > In SortMergeJoin(with wholeStageCodegen), an optimization already exists: if > the left table of a partition is empty then there is no need to read the > right table of this corresponding partition. This benefits the case in which > many partitions of left table is empty and the right table is big. > While in the code path without wholeStageCodegen, this optimization doesn't > happen. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
