GitHub user kiszk opened a pull request:
https://github.com/apache/spark/pull/20636
[SPARK-23415][SQL][TEST] Make behavior of BufferHolderSparkSubmitSuite
correct and stable
## What changes were proposed in this pull request?
This PR addresses two issues in `BufferHolderSparkSubmitSuite`.
1. While `BufferHolderSparkSubmitSuite` tried to allocate a large object
several times, it actually allocated an object once and reused the object.
2. `BufferHolderSparkSubmitSuite` may fail due to timeout
To assign a small object before allocating a large object each time solved
issue 1 by avoiding reuse.
To increasing heap size from 4g to 7g **probably** solved issue 2. It can
also avoid OOM after fixing issue 1.
## How was this patch tested?
Updated existing `BufferHolderSparkSubmitSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kiszk/spark SPARK-23415
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20636.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20636
----
commit 39a715c52ac4055b3a54e221d692039b8ac20e97
Author: Kazuaki Ishizaki <ishizaki@...>
Date: 2018-02-19T08:55:11Z
initial commit
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
