GitHub user CodingCat opened a pull request:
https://github.com/apache/spark/pull/5098
[SPARK-6422][STREAMING] support customized actor system for actor-based
receiver
https://issues.apache.org/jira/browse/SPARK-6422
To disable the Akka's fault detection system, we currently set large
default values for Akka' transport failure detector threshold and heartbeat
interval, and discourage users to customize these values..
Meanwhile, we are trying to eliminate the dependency to Akka, e.g. we are
trying to implement a general RPC interface
(https://github.com/apache/spark/pull/4588), etc.
Based on the above facts, enabling a customized Akka system for actor-based
receiver has at least two benefits:
1. users can easily set values related to failure detector which are more
practical in general Akka usage scenario;
2. reduce the dependency level to Akka, (in future, we can even move
Actor-based receiver to external project)
This patch proposes to enable the user to be able to configure the
customized Akka system by passing `spark.streaming.receiver.akka.*`
parameters...if these parameters do not exist, then we fall back to the Spark's
Actor System
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/CodingCat/spark SPARK-6422
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5098.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5098
----
commit 2f7358b70a027472f1e1f7bbb2db4fd3c95782d3
Author: CodingCat <[email protected]>
Date: 2015-03-20T01:25:37Z
support customized actor system for actor-based receiver
commit 0bffc4d4853edee5b846506274e43a8ecf134c01
Author: CodingCat <[email protected]>
Date: 2015-03-20T01:30:08Z
the user can also use Spark's actor system
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
