[
https://issues.apache.org/jira/browse/SPARK-19802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893580#comment-15893580
]
Saisai Shao commented on SPARK-19802:
-------------------------------------
Spark's {{ApplicationHistoryProvider}} is pluggable, user could implement their
own provider and plug into Spark's history server. So you could implement a
{{HistoryProvider}} you wanted out of Spark.
>From your description, this is more like a Hadoop ATS (Hadoop application
>timeline server). We have an implementation of Timeline based history provider
>for Spark's history server. The main feature is like what you mentioned query
>through TCP, get the event and display on UI.
> Remote History Server
> ---------------------
>
> Key: SPARK-19802
> URL: https://issues.apache.org/jira/browse/SPARK-19802
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Ben Barnard
>
> Currently the history server expects to find history in a filesystem
> somewhere. It would be nice to have a history server that listens for
> application events on a TCP port, and have a EventLoggingListener that sends
> events to the listening history server instead of writing to a file. This
> would allow the history server to show up-to-date history for past and
> running jobs in a cluster environment that lacks a shared filesystem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
