GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/11031
[SPARK-7997][Core]Add rpcEnv.awaitTermination() back to SparkEnv
`rpcEnv.awaitTermination()` was not added in #10854 because some Streaming
Python tests hung forever.
This patch fixed the hung issue and added rpcEnv.awaitTermination() back to
SparkEnv.
Previously, Streaming Kafka Python tests shutdowns the zookeeper server
before stopping StreamingContext. Then when stopping StreamingContext,
KafkaReceiver may be hung due to
https://issues.apache.org/jira/browse/KAFKA-601, hence, some thread of RpcEnv's
Dispatcher cannot exit and rpcEnv.awaitTermination is hung.The patch just
changed the shutdown order to fix it.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark awaitTermination
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11031.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11031
----
commit 224916fd0fbaad80a15edc0834806715c0955024
Author: Shixiong Zhu <[email protected]>
Date: 2016-01-29T00:59:14Z
Add rpcEnv.awaitTermination() back to SparkEnv
`rpcEnv.awaitTermination()` was not added in #10854 because some Streaming
Python tests hung forever.
This patch fixed the hung issue and added rpcEnv.awaitTermination() back to
SparkEnv.
Previously, Streaming Kafka Python tests shutdowns the zookeeper server
before stopping StreamingContext. Then when stopping StreamingContext,
KafkaReceiver may be hung due to
https://issues.apache.org/jira/browse/KAFKA-601, hence, some thread of RpcEnv's
Dispatcher cannot exit and rpcEnv.awaitTermination is hung.The patch just
changed the shutdown order to fix it.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
