Josh Rosen created SPARK-5411:
---------------------------------
Summary: Allow SparkListeners to be specified in SparkConf and
loaded when creating SparkContext
Key: SPARK-5411
URL: https://issues.apache.org/jira/browse/SPARK-5411
Project: Spark
Issue Type: New Feature
Components: Spark Core
Reporter: Josh Rosen
Assignee: Josh Rosen
It would be nice if there was a mechanism to allow SparkListeners to be
registered through SparkConf settings. This would allow monitoring frameworks
to be easily injected into Spark programs without having to modify those
programs' code.
I propose to introduce a new configuration option, {{spark.extraListeners}},
that allows SparkListeners to be specified in SparkConf and registered before
the SparkContext is created. Here is the proposed documentation for the new
option:
{quote}
A comma-separated list of classes that implement SparkListener; when
initializing SparkContext, instances of these classes will be created and
registered with Spark's listener bus. If a class has a single-argument
constructor that accepts a SparkConf, that constructor will be called;
otherwise, a zero-argument constructor will be called. If no valid constructor
can be found, the SparkContext creation will fail with an exception.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
