Shixiong Zhu created SPARK-15262:
------------------------------------
Summary: race condition in killing an executor and reregistering
an executor
Key: SPARK-15262
URL: https://issues.apache.org/jira/browse/SPARK-15262
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: Shixiong Zhu
There is a race condition when killing an executor and reregistering an
executor happen at the same time. Here is the execution steps to reproduce it.
1. master find a worker is dead
2. master tells driver to remove executor
3. driver remove executor
4. BlockManagerMasterEndpoint remove the block manager
5. executor finds it's not reigstered via heartbeat
6. executor send reregister block manager
7. register block manager
8. executor is killed by worker
9. CoarseGrainedSchedulerBackend ignores onDisconnected as this address is not
in the executor list
10. BlockManagerMasterEndpoint.blockManagerInfo contains dead block managers
As BlockManagerMasterEndpoint.blockManagerInfo contains some dead block
managers, when we unpersist a RDD, remove a broadcast, or clean a shuffle block
via a RPC endpoint of a dead block manager, we will get ClosedChannelException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
