GitHub user KaiXinXiaoLei opened a pull request:
https://github.com/apache/spark/pull/9268
[SPARK-11298] When driver sends message "GetExecutorLossReason" to AM, the
SparkContext may be stop
I get lastest code form github, and just run "bin/spark-shell --master yarn
--conf spark.dynamicAllocation.enabled=true --conf
spark.dynamicAllocation.initialExecutors=1 --conf
spark.shuffle.service.enabled=true". There is error infor:
15/10/25 12:11:02 ERROR TransportChannelHandler: Connection to
/9.96.1.113:35066 has been quiet for 120000 ms while there are outstanding
requests. Assuming connection is dead; please adjust spark.network.timeout if
this is wrong.
15/10/25 12:11:02 ERROR TransportResponseHandler: Still have 1 requests
outstanding when connection from vm113/9.96.1.113:35066 is closed
15/10/25 12:11:02 WARN NettyRpcEndpointRef: Ignore message
Failure(java.io.IOException: Connection from vm113/9.96.1.113:35066 closed)
15/10/25 12:11:02 ERROR YarnScheduler: Lost executor 1 on vm111: Slave lost
From log, when driver sends message "GetExecutorLossReason" to AM, the
error appears. From code, i think AM gets this message, should reply.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KaiXinXiaoLei/spark replayAM
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9268.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9268
----
commit cf661ded1f864d9bb75293a63f758640c847010f
Author: KaiXinXiaoLei <[email protected]>
Date: 2015-10-25T06:33:01Z
reply
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
