[jira] [Updated] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Lantao Jin (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lantao Jin updated SPARK-32129:
---
Description: 
Current, the AQE skew join only supports two tables join such as
{code}
SMJ
:-Sort
:+-Shuffle
+-Sort
 +-Shuffle
{code}
But if the plan contains a Union, the skew join handling not work:
{code}
Union
:-SMJ
:   :-Sort
:   : +-Shuffle
:   +-Sort
: +-Shuffle
+-SMJ
:   :-Sort
:   : +-Shuffle
:   +-Sort
 +-Shuffle
{code}


  was:
Current, the AQE skew join only supports two tables join such as
{code}
SMJ
:-Sort
:+-Shuffle
+-Sort
 +-Shuffle
{code}
But if the plan contains a Union, the skew join handling not work:
{code}
Union
:-SMJ
:   :-Sort
:   : +-Shuffle
:  +-Sort
: +-Shuffle
+-SMJ
:   :-Sort
:   : +-Shuffle
:  +-Sort
 +-Shuffle
{code}



> Support AQE skew join with Union
> 
>
> Key: SPARK-32129
> URL: https://issues.apache.org/jira/browse/SPARK-32129
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Lantao Jin
>Priority: Major
>
> Current, the AQE skew join only supports two tables join such as
> {code}
> SMJ
> :-Sort
> :+-Shuffle
> +-Sort
>  +-Shuffle
> {code}
> But if the plan contains a Union, the skew join handling not work:
> {code}
> Union
> :-SMJ
> :   :-Sort
> :   : +-Shuffle
> :   +-Sort
> : +-Shuffle
> +-SMJ
> :   :-Sort
> :   : +-Shuffle
> :   +-Sort
>  +-Shuffle
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Lantao Jin (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lantao Jin updated SPARK-32129:
---
Description: 
Current, the AQE skew join only supports two tables join such as
{code}
SMJ
:-Sort
:+-Shuffle
+-Sort
 +-Shuffle
{code}
But if the plan contains a Union, the skew join handling not work:
{code}
Union
:-SMJ
:   :-Sort
:   :+-Shuffle
:   +-Sort
:+-Shuffle
+-SMJ
:   :-Sort
:   :+-Shuffle
:   +-Sort
 +-Shuffle
{code}


  was:
Current, the AQE skew join only supports two tables join such as
{code}
SMJ
:-Sort
:+-Shuffle
+-Sort
 +-Shuffle
{code}
But if the plan contains a Union, the skew join handling not work:
{code}
Union
:-SMJ
:   :-Sort
:   : +-Shuffle
:   +-Sort
: +-Shuffle
+-SMJ
:   :-Sort
:   : +-Shuffle
:   +-Sort
 +-Shuffle
{code}



> Support AQE skew join with Union
> 
>
> Key: SPARK-32129
> URL: https://issues.apache.org/jira/browse/SPARK-32129
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Lantao Jin
>Priority: Major
>
> Current, the AQE skew join only supports two tables join such as
> {code}
> SMJ
> :-Sort
> :+-Shuffle
> +-Sort
>  +-Shuffle
> {code}
> But if the plan contains a Union, the skew join handling not work:
> {code}
> Union
> :-SMJ
> :   :-Sort
> :   :+-Shuffle
> :   +-Sort
> :+-Shuffle
> +-SMJ
> :   :-Sort
> :   :+-Shuffle
> :   +-Sort
>  +-Shuffle
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org