[
https://issues.apache.org/jira/browse/SPARK-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337498#comment-14337498
]
Hans van den Bogert edited comment on SPARK-3612 at 2/25/15 11:55 PM:
----------------------------------------------------------------------
If I understand this patch correctly, I shouldn't be seeing this anymore right?
{code}
15/02/25 22:52:52 WARN AkkaUtils: Error sending message in 1 attempts
java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:187)
at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:398)
{code}
On version 1.2.0
was (Author: hbogert):
If I understand this patch correctly, I shouldn't be seeing this anymore right?
{code}
15/02/25 22:52:52 WARN AkkaUtils: Error sending message in 1 attempts
java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:187)
at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:398)
{code}
> Executor shouldn't quit if heartbeat message fails to reach the driver
> ----------------------------------------------------------------------
>
> Key: SPARK-3612
> URL: https://issues.apache.org/jira/browse/SPARK-3612
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Reynold Xin
> Assignee: Sandy Ryza
> Fix For: 1.1.1, 1.2.0
>
>
> The thread started by Executor.startDriverHeartbeater can actually terminate
> the whole executor if AkkaUtils.askWithReply[HeartbeatResponse] throws an
> exception.
> I don't think we should quit the executor this way. At the very least, we
> would want to log a more meaningful exception then simply
> {code}
> 14/09/20 06:38:12 WARN AkkaUtils: Error sending message in 1 attempts
> java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> at
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.result(package.scala:107)
> at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:176)
> at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:379)
> 14/09/20 06:38:45 WARN AkkaUtils: Error sending message in 2 attempts
> java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> at
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.result(package.scala:107)
> at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:176)
> at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:379)
> 14/09/20 06:39:18 WARN AkkaUtils: Error sending message in 3 attempts
> java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> at
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.result(package.scala:107)
> at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:176)
> at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:379)
> 14/09/20 06:39:21 ERROR ExecutorUncaughtExceptionHandler: Uncaught exception
> in thread Thread[Driver Heartbeater,5,main]
> org.apache.spark.SparkException: Error sending message [message =
> Heartbeat(281,[Lscala.Tuple2;@4d9294db,BlockManagerId(281,
> ip-172-31-7-55.eu-west-1.compute.internal, 52303))]
> at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:190)
> at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:379)
> Caused by: java.util.concurrent.TimeoutException: Futures timed out after [30
> seconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> at
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.result(package.scala:107)
> at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:176)
> ... 1 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
