[ 
https://issues.apache.org/jira/browse/YARN-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14100873#comment-14100873
 ] 

Zhijie Shen commented on YARN-2033:
-----------------------------------

The test failures seem to be related to RM HA.

bq. IMO, this is not necessary as application should exist in most cases and we 
don't need to duplicated visit LevelDB twice. If application doesn't exist, we 
can throw ApplicationNotFoundException in retrieving app attempt info. Isn't it?

First of all, two queries are not duplicate: one to read the application 
entity, and the other to read the app attempt entity, and we previously 
distinguish ApplicationNotFoundException and  
ApplicationAttemptNotFoundException. It is always possible that App1 exists in 
the store with the only attempt AppAttempt1 while the user looks up for 
AppAttempt2. In this case, we know App1 is there, but AppAttempt2 isn't, so we 
will throw ApplicationAttemptNotFoundException.

Moreover, when we go on with generic history ACLs, we will anyway visit the app 
entity once to pull the user info for access check.

bq. The point here is we should check on the combination of related 
configurations and make all wrong combinations get warned. Any concern on doing 
this?

Right, so in the new patch, I've enhanced the configuration check logic, to 
make sure either the old or the new history service stack will be used, but not 
both. However, I don't cover mis config of within the scope of the old history 
service stack itself. For example, ApplicationHistoryStore -> null store while 
enabling history service. It even didn't work in the previous situation.

bq. I mean to define "hadoop.tmp.dir" in YarnConfiguration to be something 
like: HADOOP_TMP_DIR which sounds more uniform when dealing with config.

"hadoop.tmp.dir" should be part of YarnConfiguration. If it really needs to be 
added, it should be placed in CommonConfigurationKeys. However, I'm afraid it's 
not a good idea to do that, either. Let's look into the default of it.
{code}
<property>
  <name>hadoop.tmp.dir</name>
  <value>/tmp/hadoop-${user.name}</value>
  <description>A base for other temporary directories.</description>
</property>
{code}
The default comes with a param, which can not be determined upfront either. 
AFAIK, all such kind of defaults are not contained in config classes.

> Investigate merging generic-history into the Timeline Store
> -----------------------------------------------------------
>
>                 Key: YARN-2033
>                 URL: https://issues.apache.org/jira/browse/YARN-2033
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Zhijie Shen
>         Attachments: ProposalofStoringYARNMetricsintotheTimelineStore.pdf, 
> YARN-2033.1.patch, YARN-2033.2.patch, YARN-2033.3.patch, YARN-2033.4.patch, 
> YARN-2033.5.patch, YARN-2033.6.patch, YARN-2033.Prototype.patch, 
> YARN-2033_ALL.1.patch, YARN-2033_ALL.2.patch, YARN-2033_ALL.3.patch, 
> YARN-2033_ALL.4.patch
>
>
> Having two different stores isn't amicable to generic insights on what's 
> happening with applications. This is to investigate porting generic-history 
> into the Timeline Store.
> One goal is to try and retain most of the client side interfaces as close to 
> what we have today.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to