[
https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192538#comment-14192538
]
Zhijie Shen commented on SPARK-1537:
------------------------------------
bq. Spark needs both to put and read data
It's again a vague statement. Can you share your design detail, such that we
can evaluate it is really necessary?
And what is the actual way of visualizing data? And integration work is not
just single bug fix patch, we can divide work into a sequent of sub tasks, and
the first step is to enable Spark job to be able to putting the data into the
timeline server. By doing this, not only Spark's only web front can visualize
job history, it also enable the third-party tools to do Spark job analysis too.
bq. I'm not sure why you say it's security-related since there nothing
security-related in the example code I posted.
I said "According to the exception, the user doesn't pass the authorization for
some reason." If you don't agree on it, please post your investigation on
YARN-2444, YARN folks will help you on this issue.
bq. if something doesn't work in 2.5 but works in 2.6,
No matter the integration with timeline service, Spark on YARN is picking
Hadoop versions now. It doesn't make sense to ask for a feature by using an
early version that hasn't it.
> Add integration with Yarn's Application Timeline Server
> -------------------------------------------------------
>
> Key: SPARK-1537
> URL: https://issues.apache.org/jira/browse/SPARK-1537
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Reporter: Marcelo Vanzin
> Assignee: Marcelo Vanzin
>
> It would be nice to have Spark integrate with Yarn's Application Timeline
> Server (see YARN-321, YARN-1530). This would allow users running Spark on
> Yarn to have a single place to go for all their history needs, and avoid
> having to manage a separate service (Spark's built-in server).
> At the moment, there's a working version of the ATS in the Hadoop 2.4 branch,
> although there is still some ongoing work. But the basics are there, and I
> wouldn't expect them to change (much) at this point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
