date:20140222


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909299#comment-13909299
 ] 

Hadoop QA commented on YARN-1749:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12630467/YARN-1687%2BYARN-1732%2BYARN-1749.2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 11 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/3156//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3156//console

This message is automatically generated.

 Misc fixes of AHS configurations and related behavior
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1470) Add audience annotation to MiniYARNCluster


[ 
https://issues.apache.org/jira/browse/YARN-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909333#comment-13909333
 ] 

Hudson commented on YARN-1470:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #489 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/489/])
YARN-1470. Add audience annotations to MiniYARNCluster. (Anubhav Dhoot via 
kasha) (kasha: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570708)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java


 Add audience annotation to MiniYARNCluster
 --

 Key: YARN-1470
 URL: https://issues.apache.org/jira/browse/YARN-1470
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Anubhav Dhoot
  Labels: newbie
 Fix For: 2.4.0

 Attachments: YARN-1470.001.patch


 We should make it clear whether this is a public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1470) Add audience annotation to MiniYARNCluster


[ 
https://issues.apache.org/jira/browse/YARN-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909371#comment-13909371
 ] 

Hudson commented on YARN-1470:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk #1681 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1681/])
YARN-1470. Add audience annotations to MiniYARNCluster. (Anubhav Dhoot via 
kasha) (kasha: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570708)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java


 Add audience annotation to MiniYARNCluster
 --

 Key: YARN-1470
 URL: https://issues.apache.org/jira/browse/YARN-1470
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Anubhav Dhoot
  Labels: newbie
 Fix For: 2.4.0

 Attachments: YARN-1470.001.patch


 We should make it clear whether this is a public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (YARN-1753) FileSystemApplicationHistoryStore#HistoryFileReader#next() should check return value of dis.read()

2014-02-22 Thread Ted Yu (JIRA)

Ted Yu created YARN-1753:


 Summary: 
FileSystemApplicationHistoryStore#HistoryFileReader#next() should check return 
value of dis.read()
 Key: YARN-1753
 URL: https://issues.apache.org/jira/browse/YARN-1753
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor


Here is related code:
{code}
  byte[] value = new byte[entry.getValueLength()];
  dis.read(value);
{code}
entry.getValueLength() bytes are expected to be read.
The return value from dis.read() should be checked against value length.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1470) Add audience annotation to MiniYARNCluster


[ 
https://issues.apache.org/jira/browse/YARN-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909401#comment-13909401
 ] 

Hudson commented on YARN-1470:
--

SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1706 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1706/])
YARN-1470. Add audience annotations to MiniYARNCluster. (Anubhav Dhoot via 
kasha) (kasha: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570708)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java


 Add audience annotation to MiniYARNCluster
 --

 Key: YARN-1470
 URL: https://issues.apache.org/jira/browse/YARN-1470
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Anubhav Dhoot
  Labels: newbie
 Fix For: 2.4.0

 Attachments: YARN-1470.001.patch


 We should make it clear whether this is a public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

2014-02-22 Thread Ming Ma (JIRA)

[
https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909440#comment-13909440
]

Ming Ma commented on YARN-221:
--

One thing to add we need the definition and policy on how to handle those tasks
that are in the finishing state and MR AM ends up stopping them as they don't
exit by themselves. From customers's point of view, the task is considered to
be successful. For log aggregation point of view, if we want to aggregate only
failed task, then MR AM still needs to tell NM not to do log aggregation for
such kind of tasks. Maybe this isn't important if we believe most of the tasks
will exit by themselves. But it is still useful to provide the proper
definition and policy for it.

NM should provide a way for AM to tell it not to aggregate logs.

Key: YARN-221
URL: https://issues.apache.org/jira/browse/YARN-221
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Robert Joseph Evans
Assignee: Chris Trezzo
Attachments: YARN-221-trunk-v1.patch

The NodeManager should provide a way for an AM to tell it that either the
logs should not be aggregated, that they should be aggregated with a high
priority, or that they should be aggregated but with a lower priority. The
AM should be able to do this in the ContainerLaunch context to provide a
default value, but should also be able to update the value when the container
is released.
This would allow for the NM to not aggregate logs in some cases, and avoid
connection to the NN at all.

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1748) hadoop-yarn-server-tests packages core-site.xml breaking downstream tests

2014-02-22 Thread Sravya Tirukkovalur (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sravya Tirukkovalur updated YARN-1748:
--

Summary: hadoop-yarn-server-tests packages core-site.xml breaking 
downstream tests  (was: hadoop-mapreduce-client-jobclient packages 
core-site.xml breaking downstream tests)

 hadoop-yarn-server-tests packages core-site.xml breaking downstream tests
 -

 Key: YARN-1748
 URL: https://issues.apache.org/jira/browse/YARN-1748
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Sravya Tirukkovalur

 Jars should not package config files, as this might come into the classpaths 
 of clients causing the clients to break.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1748) hadoop-yarn-server-tests packages core-site.xml breaking downstream tests


 [ 
https://issues.apache.org/jira/browse/YARN-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-1748:
--

Assignee: Vinod Kumar Vavilapalli

We need those conf files for YARN tests themselves. I can see why downstream 
projects can break. I think the real problem is that MiniYARNCluster is under a 
test jar (under src/tests). I think that moving class to regular code 
(src/java) will avoid this.

IOW, downstream projects will need to start depending on 
hadoop-yarn-server-tests-2.3.0.jar instead of 
hadoop-yarn-server-tests-2.3.0-tests.jar if we change that. At which point 
whether we have core-site in our test code or not shouldn't matter.

This is kind of an incompatible change w.r.t jar dependencies. I'd like to 
gather opinions before we do this.


 hadoop-yarn-server-tests packages core-site.xml breaking downstream tests
 -

 Key: YARN-1748
 URL: https://issues.apache.org/jira/browse/YARN-1748
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Sravya Tirukkovalur
Assignee: Vinod Kumar Vavilapalli

 Jars should not package config files, as this might come into the classpaths 
 of clients causing the clients to break.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1363) Get / Cancel / Renew delegation token api should be non blocking

2014-02-22 Thread Jian He (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909510#comment-13909510
 ] 

Jian He commented on YARN-1363:
---

Haven’t looked at the patch deeply, some thoughts
- we don’t need a separate api getIsObtained in GetDelegationTokenResponse, 
setting the token as null can be leveraged to indicate not yet obtained.
- what will happen if one operation is in the progress like getDelegationToken 
while in the meantime another operation like cancelDelegationToken comes in ?
  Thinking whether it’s good to have a state machine per delegation token. We 
have 6 events GET,CANCEL,RENEW from client, and STORED, UPDATED, REMOVED from 
state store; 3 states, NEW, PROGRESS(or getting, canceling, renewing if needed) 
and FINISHED. We should think about all the possible combinations.

 Get / Cancel / Renew delegation token api should be non blocking
 

 Key: YARN-1363
 URL: https://issues.apache.org/jira/browse/YARN-1363
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Omkar Vinit Joshi
Assignee: Zhijie Shen
 Attachments: YARN-1363.1.patch, YARN-1363.2.patch, YARN-1363.3.patch, 
 YARN-1363.4.patch, YARN-1363.5.patch, YARN-1363.6.patch, YARN-1363.7.patch


 Today GetDelgationToken, CancelDelegationToken and RenewDelegationToken are 
 all blocking apis.
 * As a part of these calls we try to update RMStateStore and that may slow it 
 down.
 * Now as we have limited number of client request handlers we may fill up 
 client handlers quickly.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-745) Move UnmanagedAMLauncher to yarn client package

2014-02-22 Thread Bikas Saha (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909521#comment-13909521
 ] 

Bikas Saha commented on YARN-745:
-

Sorry for late response. Please go ahead in case you havent already.

 Move UnmanagedAMLauncher to yarn client package
 ---

 Key: YARN-745
 URL: https://issues.apache.org/jira/browse/YARN-745
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Bikas Saha
Assignee: Bikas Saha
 Fix For: 2.3.0


 Its currently sitting in yarn applications project which sounds wrong. client 
 project sounds better since it contains the utilities/libraries that clients 
 use to write and debug yarn applications.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1732) Change types of related entities and primary filters in ATSEntity


[ 
https://issues.apache.org/jira/browse/YARN-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909524#comment-13909524
 ] 

Vinod Kumar Vavilapalli commented on YARN-1732:
---

Checking this in..

 Change types of related entities and primary filters in ATSEntity
 -

 Key: YARN-1732
 URL: https://issues.apache.org/jira/browse/YARN-1732
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Attachments: YARN-1732.1.patch, YARN-1732.2.patch, YARN-1732.3.patch


 The current types MapString, ListString relatedEntities and MapString, 
 Object primaryFilters have issues.  The ListString value of the related 
 entities map could have multiple identical strings in it, which doesn't make 
 sense. A more major issue is that we cannot allow primary filter values to be 
 overwritten, because otherwise we will be unable to find those primary filter 
 entries when we want to delete an entity (without doing a nearly full scan).
 I propose changing related entities to MapString, SetString and primary 
 filters to MapString, SetObject.  The basic methods to add primary 
 filters and related entities are of the form add(key, value) and will not 
 need to change.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1687) Refactoring timeline classes to remove app related words


[ 
https://issues.apache.org/jira/browse/YARN-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909529#comment-13909529
 ] 

Vinod Kumar Vavilapalli commented on YARN-1687:
---

The patch looks good to me, +1.

YARN-1732 is in too. Checking this in now..

 Refactoring timeline classes to remove app related words
 --

 Key: YARN-1687
 URL: https://issues.apache.org/jira/browse/YARN-1687
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732.1.patch, 
 YARN-1687+YARN-1732.2.patch, YARN-1687+YARN-1732.3.patch, YARN-1687.1.patch, 
 YARN-1687.2.patch, YARN-1687.3.patch, YARN-1687.4.patch


 Remove ATS prefix, change package name, fix javadoc and so on



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1748) hadoop-yarn-server-tests packages core-site.xml breaking downstream tests

2014-02-22 Thread Sravya Tirukkovalur (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909530#comment-13909530
 ] 

Sravya Tirukkovalur commented on YARN-1748:
---

Thanks for having a look Vinod, yes I agree it would be better to pull in the 
mini cluster to src/java. Another thing we could do is to simply have an 
exclude clause, something like:
https://issues.apache.org/jira/browse/MAPREDUCE-5554

 hadoop-yarn-server-tests packages core-site.xml breaking downstream tests
 -

 Key: YARN-1748
 URL: https://issues.apache.org/jira/browse/YARN-1748
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Sravya Tirukkovalur
Assignee: Vinod Kumar Vavilapalli

 Jars should not package config files, as this might come into the classpaths 
 of clients causing the clients to break.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1732) Change types of related entities and primary filters in ATSEntity


[ 
https://issues.apache.org/jira/browse/YARN-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909533#comment-13909533
 ] 

Hudson commented on YARN-1732:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5211 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5211/])
YARN-1732. Changed types of related-entities and primary-filters in the 
timeline-service to be sets instead of maps. Contributed by Billie Rinaldi. 
(vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570914)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/GenericObjectMapper.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/LeveldbApplicationTimelineStore.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/MemoryApplicationTimelineStore.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStoreTestUtils.java


 Change types of related entities and primary filters in ATSEntity
 -

 Key: YARN-1732
 URL: https://issues.apache.org/jira/browse/YARN-1732
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Fix For: 2.4.0

 Attachments: YARN-1732.1.patch, YARN-1732.2.patch, YARN-1732.3.patch


 The current types MapString, ListString relatedEntities and MapString, 
 Object primaryFilters have issues.  The ListString value of the related 
 entities map could have multiple identical strings in it, which doesn't make 
 sense. A more major issue is that we cannot allow primary filter values to be 
 overwritten, because otherwise we will be unable to find those primary filter 
 entries when we want to delete an entity (without doing a nearly full scan).
 I propose changing related entities to MapString, SetString and primary 
 filters to MapString, SetObject.  The basic methods to add primary 
 filters and related entities are of the form add(key, value) and will not 
 need to change.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1687) Refactoring timeline classes to remove app related words


[ 
https://issues.apache.org/jira/browse/YARN-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909535#comment-13909535
 ] 

Hudson commented on YARN-1687:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5212 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5212/])
YARN-1687. Renamed user-facing records for the timeline-service to be simply 
named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
(vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570922)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineEntities.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineEntity.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineEvent.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineEvents.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelinePutResponse.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/package-info.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/TimelineClient.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestTimelineClient.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/TimelineUtils.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/timeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/timeline/TimelineUtils.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/timeline/package-info.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/timeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/timeline/TestTimelineRecords.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/EntityIdentifier.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/GenericObjectMapper.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/LeveldbTimelineStore.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/MemoryTimelineStore.java

[jira] [Updated] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


 [ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-1749:
--

Summary: Review AHS configs and sycn them up with the timeline-service 
configs  (was: Misc fixes of AHS configurations and related behavior)

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs

2014-02-22 Thread Zhijie Shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhijie Shen updated YARN-1749:
--

Attachment: YARN-1749.4.patch

bq. We've gotten rid of history-writer.* in YarnConfiguration, but 
yarn-default.xml is not yet updated regarding the same.

Removed in yarn-default.xml as well, and move app-history.enable to timeline 
config section.

bq. We still have history-writer.multi-threaded-dispatcher.pool-size. Which 
seems fine, but let me know what you think.

Yes, we should have it as it reflect a writer class in RM

bq. Let's mark all the configs related to app-history as @Private for now. As 
we merge generic history with the timeline-service, some of it will change.

Done with adding the comment as well

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909570#comment-13909570
 ] 

Hadoop QA commented on YARN-1749:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12630501/YARN-1749.4.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/3157//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3157//console

This message is automatically generated.

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


 [ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-1749:
--

Attachment: YARN-1749.5.patch

Same patch as before but with minor updates
 - One minor update - renaming *.app-history.* to 
*.generic-application-history.* to be clear.
 - Also (hard to point out in JIRA) reordering yarn-default.xml settings to 
group them better.

Will commit it if Jenkins says okay..

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs

2014-02-22 Thread Zhijie Shen (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909614#comment-13909614
 ] 

Zhijie Shen commented on YARN-1749:
---

Hi Vinod, thanks for helping with new patch. +1 for the 2 updates. I just find 
one more bug in 
yarn.timeline-service.generic-application-history.fs-history-store.uri, I 
uploaded a new patch to fix it.

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs

2014-02-22 Thread Zhijie Shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhijie Shen updated YARN-1749:
--

Attachment: YARN-1749.6.patch

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909619#comment-13909619
 ] 

Hadoop QA commented on YARN-1749:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12630514/YARN-1749.5.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The following test timeouts occurred in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/3158//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3158//console

This message is automatically generated.

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909628#comment-13909628
 ] 

Hadoop QA commented on YARN-1749:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12630519/YARN-1749.6.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/3159//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3159//console

This message is automatically generated.

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sycn them up with the timeline-service configs


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909631#comment-13909631
 ] 

Vinod Kumar Vavilapalli commented on YARN-1749:
---

Alright, that looks great. Time to check this in..

 Review AHS configs and sycn them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1749) Review AHS configs and sync them up with the timeline-service configs


 [ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-1749:
--

Summary: Review AHS configs and sync them up with the timeline-service 
configs  (was: Review AHS configs and sycn them up with the timeline-service 
configs)

 Review AHS configs and sync them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Fix For: 2.4.0

 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1749) Review AHS configs and sync them up with the timeline-service configs


[ 
https://issues.apache.org/jira/browse/YARN-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909638#comment-13909638
 ] 

Hudson commented on YARN-1749:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5214 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5214/])
YARN-1749. Updated application-history related configs to reflect the latest 
reality and to be consistently named. Contributed by Zhijie Shen. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570948)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AHSClientImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryClientService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/FileSystemApplicationHistoryStore.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestFileSystemApplicationHistoryStore.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ahs/RMApplicationHistoryWriter.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/ahs/TestRMApplicationHistoryWriter.java


 Review AHS configs and sync them up with the timeline-service configs
 -

 Key: YARN-1749
 URL: https://issues.apache.org/jira/browse/YARN-1749
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
 Fix For: 2.4.0

 Attachments: YARN-1687+YARN-1732+YARN-1749.1.patch, 
 YARN-1687+YARN-1732+YARN-1749.2.patch, YARN-1749.1.patch, YARN-1749.2.patch, 
 YARN-1749.3.patch, YARN-1749.4.patch, YARN-1749.5.patch, YARN-1749.6.patch


 We need to:
 1. Review the configuration names and default values
 2. Combine the two store class configurations
 Some other thoughts:
 1. Maybe we don't need null implementation of ApplicationHistoryStore any more
 2. Maybe if yarn.ahs.enabled = false, we should stop AHS web server returning 
 historic information



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1742) Fix javadoc of parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION


 [ 
https://issues.apache.org/jira/browse/YARN-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-1742:
--

Priority: Trivial  (was: Minor)

+1 for the trivial patch, checking this in.

 Fix javadoc of parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION
 --

 Key: YARN-1742
 URL: https://issues.apache.org/jira/browse/YARN-1742
 Project: Hadoop YARN
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.2.0
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Priority: Trivial
  Labels: newbie
 Attachments: YARN-1742.patch


 In YarnConfiguration.java, 
 {code}
   /**
* By default, at least 5% of disks are to be healthy to say that the node
* is healthy in terms of disks.
*/
   public static final float DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION
 = 0.25F;
 {code}
 25% is the correct.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-986) YARN should use cluster-id as token service address

2014-02-22 Thread Karthik Kambatla (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909651#comment-13909651
 ] 

Karthik Kambatla commented on YARN-986:
---

Got this working. Will cleanup the patch and post later tonight. 

 YARN should use cluster-id as token service address
 ---

 Key: YARN-986
 URL: https://issues.apache.org/jira/browse/YARN-986
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Karthik Kambatla
Priority: Blocker
 Attachments: yarn-986-prelim-0.patch


 This needs to be done to support non-ip based fail over of RM. Once the 
 server sets the token service address to be this generic ClusterId/ServiceId, 
 clients can translate it to appropriate final IP and then be able to select 
 tokens via TokenSelectors.
 Some workarounds for other related issues were put in place at YARN-945.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1742) Fix javadoc of parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION


[ 
https://issues.apache.org/jira/browse/YARN-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909652#comment-13909652
 ] 

Hudson commented on YARN-1742:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5215 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5215/])
YARN-1742. Fixed javadoc of configuration parameter 
DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION. Contributed by Akira Ajisaka. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1570956)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java


 Fix javadoc of parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION
 --

 Key: YARN-1742
 URL: https://issues.apache.org/jira/browse/YARN-1742
 Project: Hadoop YARN
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.2.0
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Priority: Trivial
  Labels: newbie
 Fix For: 2.4.0

 Attachments: YARN-1742.patch


 In YarnConfiguration.java, 
 {code}
   /**
* By default, at least 5% of disks are to be healthy to say that the node
* is healthy in terms of disks.
*/
   public static final float DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION
 = 0.25F;
 {code}
 25% is the correct.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1686) NodeManager.resyncWithRM() does not handle exception which cause NodeManger to Hang.

2014-02-22 Thread Jian He (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909667#comment-13909667
 ] 

Jian He commented on YARN-1686:
---

Good catch! thanks for working on this.

 NodeManager.resyncWithRM() does not handle exception which cause NodeManger 
 to Hang.
 

 Key: YARN-1686
 URL: https://issues.apache.org/jira/browse/YARN-1686
 Project: Hadoop YARN
  Issue Type: Bug
  Components: nodemanager
Affects Versions: 2.3.0
Reporter: Rohith
Assignee: Rohith
 Fix For: 3.0.0

 Attachments: YARN-1686.1.patch


 During start of NodeManager,if registration with resourcemanager throw 
 exception then nodemager shutdown happens. 
 Consider case where NM-1 is registered with RM. RM issued Resync to NM. If 
 any exception thrown in resyncWithRM (starts new thread which does not 
 handle exception) during RESYNC evet, then this thread is lost. NodeManger 
 enters hanged state. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-986) YARN should use cluster-id as token service address

2014-02-22 Thread Karthik Kambatla (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla updated YARN-986:
--

Attachment: yarn-986-1.patch

Uploading a patch that works. I am able to run Oozie jobs successfully on a 
secure cluster with the patch applied.

Basic approach - instead of using one RM's address in the RMDTToken's service, 
add both RM's addresses in the same service. Alternatively, we could generate 
two tokens as HDFS does, but a single token with an augment service name seems 
a simpler approach.

Compatibility - the token service will have two RMs only when HA is enabled. 
So, on a non-HA cluster, this is no different from how things work currently. 

 YARN should use cluster-id as token service address
 ---

 Key: YARN-986
 URL: https://issues.apache.org/jira/browse/YARN-986
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Karthik Kambatla
Priority: Blocker
 Attachments: yarn-986-1.patch, yarn-986-prelim-0.patch


 This needs to be done to support non-ip based fail over of RM. Once the 
 server sets the token service address to be this generic ClusterId/ServiceId, 
 clients can translate it to appropriate final IP and then be able to select 
 tokens via TokenSelectors.
 Some workarounds for other related issues were put in place at YARN-945.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-986) YARN should use cluster-id as token service address

2014-02-22 Thread Karthik Kambatla (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909678#comment-13909678
 ] 

Karthik Kambatla commented on YARN-986:
---

[~vinodkv], [~bikassaha] - if you guys think this is a reasonable approach, I 
ll go ahead and update the JIRA title and description accordingly. Thanks.

 YARN should use cluster-id as token service address
 ---

 Key: YARN-986
 URL: https://issues.apache.org/jira/browse/YARN-986
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Karthik Kambatla
Priority: Blocker
 Attachments: yarn-986-1.patch, yarn-986-prelim-0.patch


 This needs to be done to support non-ip based fail over of RM. Once the 
 server sets the token service address to be this generic ClusterId/ServiceId, 
 clients can translate it to appropriate final IP and then be able to select 
 tokens via TokenSelectors.
 Some workarounds for other related issues were put in place at YARN-945.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-986) YARN should use cluster-id as token service address