[
https://issues.apache.org/jira/browse/SPARK-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michal Malohlava updated SPARK-3270:
------------------------------------
Summary: Spark API for Application Extensions (was: Spark API for Embedded
Applications)
> Spark API for Application Extensions
> ------------------------------------
>
> Key: SPARK-3270
> URL: https://issues.apache.org/jira/browse/SPARK-3270
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Michal Malohlava
>
> At the begining, let's clarify my motivation - I would like to extend Spark
> platform by an embedded application (e.g., monitoring network performance in
> the context of selected applications) which will be launched on particular
> nodes in cluster with their launch.
> Nevertheless, I do not want to modify Spark code directly and hardcode my
> code in, but I would prefer to provide a jar which would be registered and
> launched by Spark itself.
> Hence, to support such 3rd party applications (aka "extesions"/"plugins")
> Spark platform should provide at least:
> - an API to register an extension
> - an API to register a "service" (meaning provided functionality)
> - well-defined points in Spark infrastructure which can be enriched/hooked
> by extension
> - in master/worker lifecycle
> - in applications lifecycle
> - in RDDs lifecycle
> - monitoring/reporting
> - ...
> - a way of deploying extension (for example, simply putting the extension
> on classpath and using Java service interface)
> In this context, I do not mean reinventing OSGi (or another plugin platform)
> but it can serve as a good starting point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
