GitHub user wujianping10043419 opened a pull request:
https://github.com/apache/spark/pull/20530
[SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and redundant type
determination for ShuffleManager Object
## What changes were proposed in this pull request?
org.apache.spark.sql.execution.exchange.ShuffleExchangeExec:
There is a nested "if or else" branch within the
"needtocopyobjectsbeforguffle()" function.
The \<sortBasedShufffleOn\> condition in the first layer "if" has the same
value as the \<bypassIsSupported\> condition in the second layer "if", that is,
\<bypassIsSupported\> must be true when \<sortBasedShufffleOn\> is true.
In addition, the \<byPassMergeThreshold\> condition will be used in the
second layer "if" and should not be calculated until needed.
It's to remove the \<bypassIsSupported\> condition in the second layer "
if" and to move the \<bypassmergethreshold\> calculation backward
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wujianping10043419/spark-1 wujp-0107-home
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20530.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20530
----
commit db50cccd1e3fd34478fc72500917211f9b3bf800
Author: wujianping10043419 <wu.jianping@...>
Date: 2018-02-07T10:59:50Z
[SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and redundant type
determination for ShuffleManager Object
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
