GitHub user heary-cao opened a pull request:
https://github.com/apache/spark/pull/19069
[MINOR][SQL][TEST]Test shuffle hash join while is not expected
## What changes were proposed in this pull request?
igore("shuffle hash join") is to shuffle hash join to test _case class
ShuffledHashJoinExec_.
But when you 'ignore' -> 'test', the test is _case class
BroadcastHashJoinExec_.
Before modified, as a result ofï¼canBroadcast is true.
Print information in _canBroadcast(plan: LogicalPlan)_
```
canBroadcast plan.stats.sizeInBytes:6710880
canBroadcast conf.autoBroadcastJoinThreshold:10000000
```
After modified, plan.stats.sizeInBytes is 11184808.
Print information in _canBuildLocalHashMap(plan: LogicalPlan)_
and _muchSmaller(a: LogicalPlan, b: LogicalPlan)_ :+1:
```
canBuildLocalHashMap plan.stats.sizeInBytes:11184808
canBuildLocalHashMap conf.autoBroadcastJoinThreshold:10000000
canBuildLocalHashMap conf.numShufflePartitions:2
```
```
muchSmaller a.stats.sizeInBytes * 3:33554424
muchSmaller b.stats.sizeInBytes:33554432
```
## How was this patch tested?
existing test case.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/heary-cao/spark shuffle_hash_join
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19069.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19069
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
