如题,下面给出了一个异常栈。

有时候提交任务也会导致flink-webui类似卡死一样大概几十秒。任务包50MB左右,提交到远程集群,提交毕竟慢,需要几十秒,快1min提交。


13:10:54.136 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.JobDetailsHandler - Unhandled
exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:10:54.140 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler
- Unhandled exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:10:54.141 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointConfigHandler
- Unhandled exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:11:04.225 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointConfigHandler
- Unhandled exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:11:04.226 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.JobDetailsHandler - Unhandled
exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:11:04.226 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler
- Unhandled exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)13:11:07.205 [flink-scheduler-1] ERROR
org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Unhandled
exception.akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka.tcp://
[email protected]:2000/user/dispatcher#-1139504499]]
after [10000 ms]. Message of type
[org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical
reason for `AskTimeoutException` is that the recipient actor didn't send a
reply.    at
akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:635)
   at
akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:650)    at
akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)    at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:870)
   at
scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)    at
scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)    at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:868)
   at
akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:279)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:283)
   at
akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:235)
   at
java.lang.Thread.run(Thread.java:748)

回复