[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-08-23 Thread viktorklang
Github user viktorklang commented on the issue:

https://github.com/apache/spark/pull/21342
  
Scala 2.13.0-M5 and forward has improved handling of InterruptedExceptions 
and fatal errors should be propagated, I'd welcome feedback on the new logic 
when possible. Thank you!

https://github.com/scala/bug/issues/9554


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
Thanks for merging !


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread gatorsmile
Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/21342
  
Thanks! Merged to master.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread JoshRosen
Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/21342
  
Updated changes LGTM. Thanks for working on this!


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91039/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #91039 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91039/testReport)**
 for PR 21342 at commit 
[`f5a5a81`](https://github.com/apache/spark/commit/f5a5a812ee32994244d29a27ddeb5730748cd212).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3508/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #91039 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91039/testReport)**
 for PR 21342 at commit 
[`f5a5a81`](https://github.com/apache/spark/commit/f5a5a812ee32994244d29a27ddeb5730748cd212).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-23 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
Thanks a gain for comments !
I updated this pr and added small defensive logic in 
SparkUncaughtExceptionHandler.scala.
Please take another look ~


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-22 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
https://issues.scala-lang.org/browse/SI-9554?orig=1 is still "OPEN", not 
sure which scala version can fix this bug.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-21 Thread JoshRosen
Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/21342
  
Thanks for the updates. The net change / scope of changes have been 
significantly reduced here, so I feel that this change is a lot less risky now.

I left only one nitpicky comment at 
https://github.com/apache/spark/pull/21342/files#r189753488 worrying about 
potential future risks from people coming along and writing new code throwing 
`SparkFatalException` in a context where it can bubble up to the uncaught 
exception handler. If we want to be super defensive we could add some logic at 
https://github.com/apache/spark/blob/32447079e9d0fa9f7e180b94ecac19091b6af1ab/core/src/main/scala/org/apache/spark/util/SparkUncaughtExceptionHandler.scala#L42
 to also catch a `SparkFatalException` on top of an OOM and treat that as an 
OOM. Debatable if we want to do that, but it's a great way of addressing 
@gatorsmile's comment at 
https://github.com/apache/spark/pull/21342#discussion_r189627101 and avoids 
future breakage.

Otherwise, LGTM.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-21 Thread cloud-fan
Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/21342
  
LGTM, which scala version has fixed this bug?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90853/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90853 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90853/testReport)**
 for PR 21342 at commit 
[`bd24746`](https://github.com/apache/spark/commit/bd2474672c1090ed748182e6d89c4bec7f940328).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3384/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90853 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90853/testReport)**
 for PR 21342 at commit 
[`bd24746`](https://github.com/apache/spark/commit/bd2474672c1090ed748182e6d89c4bec7f940328).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-20 Thread kiszk
Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/21342
  
retest this please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3380/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90847/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90847 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90847/testReport)**
 for PR 21342 at commit 
[`bd24746`](https://github.com/apache/spark/commit/bd2474672c1090ed748182e6d89c4bec7f940328).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
@kiszk @cloud-fan 
Thanks a  lot for comments.
I tested manually and found that the bug exists in all fatal throwable.
In current change, I catch all fatal throwable and wrap in 
SparkFatalException,ThreadUtils will catch SparkFatalException and re-throw 
the wrapped throwable.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
I will update the pr title if the change is on the right 
direction.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90847 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90847/testReport)**
 for PR 21342 at commit 
[`bd24746`](https://github.com/apache/spark/commit/bd2474672c1090ed748182e6d89c4bec7f940328).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90846/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90846 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90846/testReport)**
 for PR 21342 at commit 
[`bb5a74b`](https://github.com/apache/spark/commit/bb5a74b1e2770c475d0639e0751de7e8bfae8e73).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public final class SparkOutOfMemoryException extends Exception `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3379/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-19 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90846 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90846/testReport)**
 for PR 21342 at commit 
[`bb5a74b`](https://github.com/apache/spark/commit/bb5a74b1e2770c475d0639e0751de7e8bfae8e73).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/21342
  
How about we create a special Exception class to wrap OOM and 
InterupttedException, and say it's only used to work around the scala bug?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-17 Thread kiszk
Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/21342
  
Basically, I agree with this two-step direction. However, I think that it 
is too risky to simply throw and catch `SparkException`. This is because other 
`SparkException` may be caught in step 2. For example, [this 
line](https://github.com/apache/spark/pull/21342/files#diff-b024f971b46c65ddb0e4f9dcd3a831f7L100)
 throws `SparkException`.

Can we throw and catch another Exception or filter the caught 
`SparkException` with certain string?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-17 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
Thanks a lot for looking into this.

The issue is that, sometimes user would configure 
`spark.sql.broadcastTimeout` as bigger value, because the `relationFuture` in 
`BroadcastExchangeExec` could cost more time.  When OOM
happens, it makes no sense user should wait until timeout.

Yes, there is implicit behavior changes in this pr. In current change we 
throw a `SparkException` and eat the `OutOfMemoryError`. But in existing code, 
when OOM happens, `java.util.concurrent.TimeoutException` is thrown but not 
`OutOfMemoryError`.

If we really want to throw a `OutOfMemoryError` here, can we do below steps?
1. When OOM, catch in future and throw `SparkException`
2. relationFuture onFailure {
  case t: SparkException => throw OutOfMemoryError()
}


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread JoshRosen
Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/21342
  
I'm also in favor of delaying for a couple of days for more detailed review 
because historically I think these types of changes have been high risk. The 
risk calculus might be a bit different if this was fixing a critical "wrong 
result" correctness bug, but it seems like this is a longstanding annoyance 
which causes either poor performance or visible job failures, so I don't see an 
extreme urgency to get this in immediately. Therefore let's take a bit of time 
to ponder things and sleep on it to be sure that we've thought through 
corner-cases (to be clear, I do think this patch is generally in a good 
direction).

Some specifics:

1. The old code had some places which purposely caught `OutOfMemoryError` 
thrown from layers of the spilling code. I do not know whether the expected 
sources of OOMs were only the throw sites modified here or whether the intent 
was also to catch OOMs from allocating too big arrays, etc. The latter would 
have been a dodgy pattern and bad idea in the first place, but I just wanted to 
note this as a potential risk for unintended / implicit behavior changes. If we 
want to be super conservative about that we could update throw sites but keep 
catch sites and extend them to catch _both_ OOM cases.
2. Should we maybe throw an `OutOfMemoryError` subclass and then 
pattern-match on our subclass in a couple of specific places? That might help 
reduce change-surface.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/21342
  
Could we hold this PR? This change sounds risky. `OutOfMemoryError` is not 
issued by Spark. If we eat it, it could cause serious outcomes.  cc @zsxwing 
@JoshRosen 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread sameeragarwal
Github user sameeragarwal commented on the issue:

https://github.com/apache/spark/pull/21342
  
LGTM, Thanks! I've seen variations of this bug in the past as well and am 
not aware of a better way to fix this issue.

```scala
scala> Await.result(Future(throw new RuntimeException("foo")), 50.seconds)
java.lang.RuntimeException: foo // regular exceptions fail fast
```

```scala
scala> Await.result(Future(throw new OutOfMemoryError("foo")), 50.seconds)
java.util.concurrent.TimeoutException: Futures timed out after [50 seconds] 
// OOME times out
```



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread kiszk
Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/21342
  
The change looks good.
Is it better to add a comment that describes why we throw `SparkException` 
instead of `OOM` with a reference to `scala/bug#9554`. In the future, we would 
throw OOM if scala fixes this issue.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/21342
  
cc @sameeragarwal  @hvanhovell @cloud-fan @jiangxb1987 
Please take a look at this when you have time.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90676/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90676 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90676/testReport)**
 for PR 21342 at commit 
[`99e5b2a`](https://github.com/apache/spark/commit/99e5b2a905fe6a628b32cf025b137917ca4ed410).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90675/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90675 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90675/testReport)**
 for PR 21342 at commit 
[`9d83f1f`](https://github.com/apache/spark/commit/9d83f1fa397e205a826c2719a3b2978dae7d09ed).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3256/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90676 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90676/testReport)**
 for PR 21342 at commit 
[`99e5b2a`](https://github.com/apache/spark/commit/99e5b2a905fe6a628b32cf025b137917ca4ed410).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3255/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21342
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21342
  
**[Test build #90675 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90675/testReport)**
 for PR 21342 at commit 
[`9d83f1f`](https://github.com/apache/spark/commit/9d83f1fa397e205a826c2719a3b2978dae7d09ed).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org