GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/20407
[SPARK-23124][SQL] Allow to disable BroadcastNestedLoopJoin fallback
## What changes were proposed in this pull request?
In JoinStrategies, currently if no better option is available, it fallbacks
to BroadcastNestedLoopJoin. This strategy can be very problematic, since it can
cause OOM. While generally this is not a big problem, in some applications like
Thriftserver this is an issue, because a failing job can cause the whole
application to go in a bad state. Thus, in these cases, it might be useful to
be able to disable this behavior and allow to fail only the jobs which can
cause it.
## How was this patch tested?
added UT
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-23124
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20407.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20407
----
commit 074c34245d300901390d2d5ed74bb69e32539b8a
Author: Marco Gaido <marcogaido91@...>
Date: 2018-01-26T12:54:29Z
[SPARK-23124][SQL] Allow to disable BroadcastNestedLoopJoin fallback
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
