[ https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720574#comment-14720574 ]
Hitesh Shah commented on YARN-3942: ----------------------------------- [~jlowe] [~rajesh.balamohan] observed that the timeline server was running out of memory in a certain scenario. In this scenario, we are using Hive-on-Tez but Hive re-uses the application to run 100s of DAGs/queries (doAs=false with perimeter security using say Ranger or Sentry). The EntityFileStore sizes a cache based on the no. of applications it can cache but in the above scenario, even a single app could be very large. Ideally, if each "dag" was in a separate file and all of its entries treated as a single cache entity - that would probably work better but making this generic enough may be a bit tricky. Any suggestions here? > Timeline store to read events from HDFS > --------------------------------------- > > Key: YARN-3942 > URL: https://issues.apache.org/jira/browse/YARN-3942 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver > Reporter: Jason Lowe > Assignee: Jason Lowe > Attachments: YARN-3942.001.patch > > > This adds a new timeline store plugin that is intended as a stop-gap measure > to mitigate some of the issues we've seen with ATS v1 while waiting for ATS > v2. The intent of this plugin is to provide a workable solution for running > the Tez UI against the timeline server on a large-scale clusters running many > thousands of jobs per day. -- This message was sent by Atlassian JIRA (v6.3.4#6332)