[ https://issues.apache.org/jira/browse/SPARK-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin resolved SPARK-11996. --------------------------------- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 1.6.0 > Executor thread dump is broken > ------------------------------ > > Key: SPARK-11996 > URL: https://issues.apache.org/jira/browse/SPARK-11996 > Project: Spark > Issue Type: Bug > Components: Spark Core > Reporter: Shixiong Zhu > Assignee: Shixiong Zhu > Priority: Critical > Fix For: 1.6.0 > > > The driver needs to know the executor listening address to send the thread > dump request. However, in Netty RPC, the executor doesn't listen to any port, > and the executor thread dump feature is broken. Here is the stack trace: > {code} > 15/11/25 18:32:20 WARN NettyRpcEndpointRef: Error sending message [message = > GetRpcHostPortForExecutor(0)] in 1 attempts > java.lang.NullPointerException > at > org.apache.spark.storage.BlockManagerMasterEndpoint$$$$$$9c99c11fa524b2acd17a791a15f5b988$$$$$getRpcHostPortForExecutor$1$$anonfun$apply$3.apply(BlockManagerMasterEndpoint.scala:399) > at > org.apache.spark.storage.BlockManagerMasterEndpoint$$$$$$9c99c11fa524b2acd17a791a15f5b988$$$$$getRpcHostPortForExecutor$1$$anonfun$apply$3.apply(BlockManagerMasterEndpoint.scala:397) > at scala.Option.map(Option.scala:145) > at > org.apache.spark.storage.BlockManagerMasterEndpoint$$$$$$9c99c11fa524b2acd17a791a15f5b988$$$$$getRpcHostPortForExecutor$1.apply(BlockManagerMasterEndpoint.scala:397) > at > org.apache.spark.storage.BlockManagerMasterEndpoint$$$$$$9c99c11fa524b2acd17a791a15f5b988$$$$$getRpcHostPortForExecutor$1.apply(BlockManagerMasterEndpoint.scala:396) > at scala.Option.flatMap(Option.scala:170) > at > org.apache.spark.storage.BlockManagerMasterEndpoint.org$apache$spark$storage$BlockManagerMasterEndpoint$$getRpcHostPortForExecutor(BlockManagerMasterEndpoint.scala:396) > at > org.apache.spark.storage.BlockManagerMasterEndpoint$$anonfun$receiveAndReply$1.applyOrElse(BlockManagerMasterEndpoint.scala:79) > at > org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:103) > at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:206) > at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:97) > at > org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:224) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > This issue doesn't appear in `local` mode, because the driver thread dump has > a different code path. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org