[
https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720574#comment-14720574
]
Hitesh Shah commented on YARN-3942:
-----------------------------------
[~jlowe] [~rajesh.balamohan] observed that the timeline server was running out
of memory in a certain scenario. In this scenario, we are using Hive-on-Tez but
Hive re-uses the application to run 100s of DAGs/queries (doAs=false with
perimeter security using say Ranger or Sentry). The EntityFileStore sizes a
cache based on the no. of applications it can cache but in the above scenario,
even a single app could be very large. Ideally, if each "dag" was in a separate
file and all of its entries treated as a single cache entity - that would
probably work better but making this generic enough may be a bit tricky.
Any suggestions here?
> Timeline store to read events from HDFS
> ---------------------------------------
>
> Key: YARN-3942
> URL: https://issues.apache.org/jira/browse/YARN-3942
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: timelineserver
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-3942.001.patch
>
>
> This adds a new timeline store plugin that is intended as a stop-gap measure
> to mitigate some of the issues we've seen with ATS v1 while waiting for ATS
> v2. The intent of this plugin is to provide a workable solution for running
> the Tez UI against the timeline server on a large-scale clusters running many
> thousands of jobs per day.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
