[jira] [Updated] (YARN-2893) AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
[ https://issues.apache.org/jira/browse/YARN-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated YARN-2893: Attachment: YARN-2893.001.patch AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream -- Key: YARN-2893 URL: https://issues.apache.org/jira/browse/YARN-2893 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Gera Shegalov Assignee: zhihai xu Attachments: YARN-2893.000.patch, YARN-2893.001.patch MapReduce jobs on our clusters experience sporadic failures due to corrupt tokens in the AM launch context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2893) AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
[ https://issues.apache.org/jira/browse/YARN-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361996#comment-14361996 ] zhihai xu commented on YARN-2893: - Hi [~jira.shegalov], I am sorry I was busy last week. I just uploaded a new patch YARN-2893.001.patch, which has two test cases: one for app submission with valid token and the other one for app submission with invalid token. The patch will reject the app with invalid tokens. Could you review it? thanks AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream -- Key: YARN-2893 URL: https://issues.apache.org/jira/browse/YARN-2893 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Gera Shegalov Assignee: zhihai xu Attachments: YARN-2893.000.patch, YARN-2893.001.patch MapReduce jobs on our clusters experience sporadic failures due to corrupt tokens in the AM launch context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2893) AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
[
https://issues.apache.org/jira/browse/YARN-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362026#comment-14362026
]
Hadoop QA commented on YARN-2893:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704616/YARN-2893.001.patch
against trunk revision bd0a9ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6962//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6962//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6962//console
This message is automatically generated.
AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
--
Key: YARN-2893
URL: https://issues.apache.org/jira/browse/YARN-2893
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: zhihai xu
Attachments: YARN-2893.000.patch, YARN-2893.001.patch
MapReduce jobs on our clusters experience sporadic failures due to corrupt
tokens in the AM launch context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3337) Provide YARN chaos monkey
[ https://issues.apache.org/jira/browse/YARN-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362050#comment-14362050 ] Steve Loughran commented on YARN-3337: -- great to hear support. I'm thinking we may want to simulate the failure causes too, to include pre-emption and out of pmem/vmem to see how apps handle that Provide YARN chaos monkey - Key: YARN-3337 URL: https://issues.apache.org/jira/browse/YARN-3337 Project: Hadoop YARN Issue Type: New Feature Components: test Affects Versions: 2.7.0 Reporter: Steve Loughran To test failure resilience today you either need custom scripts or implement Chaos Monkey-like logic in your application (SLIDER-202). Killing AMs and containers on a schedule probability is the core activity here, one that could be handled by a CLI App/client lib that does this. # entry point to have a startup delay before acting # frequency of chaos wakeup/polling # probability to AM failure generation (0-100) # probability of non-AM container kill # future: other operations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3284) Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN command
[ https://issues.apache.org/jira/browse/YARN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362134#comment-14362134 ] Xuan Gong commented on YARN-3284: - bq. For displaying application headroom for running applications( one of the point in YARN-3273), it is required to expose applicationHeadroom field in the ApplicationAttemptMetrics.java. Would you please mind adding this field in your patch that helps retrieve headroom? Added. But use the default value: Resource.newInstance(0,0). We could replace it with the real value in YARN-3273 Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN command - Key: YARN-3284 URL: https://issues.apache.org/jira/browse/YARN-3284 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-3284.1.patch, YARN-3284.1.patch, YARN-3284.2.patch, YARN-3284.3.patch Current, we have some extra metrics about the application and current attempt in RM Web UI. We should expose that information through YARN Command, too. 1. Preemption metrics 2. application outstanding resource requests 3. container locality info -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3284) Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN command
[ https://issues.apache.org/jira/browse/YARN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-3284: Attachment: YARN-3284.3.patch Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN command - Key: YARN-3284 URL: https://issues.apache.org/jira/browse/YARN-3284 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-3284.1.patch, YARN-3284.1.patch, YARN-3284.2.patch, YARN-3284.3.patch Current, we have some extra metrics about the application and current attempt in RM Web UI. We should expose that information through YARN Command, too. 1. Preemption metrics 2. application outstanding resource requests 3. container locality info -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3284) Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN command
[
https://issues.apache.org/jira/browse/YARN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362207#comment-14362207
]
Hadoop QA commented on YARN-3284:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704628/YARN-3284.3.patch
against trunk revision bd0a9ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 8 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6963//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6963//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6963//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6963//console
This message is automatically generated.
Expose more ApplicationMetrics and ApplicationAttemptMetrics through YARN
command
-
Key: YARN-3284
URL: https://issues.apache.org/jira/browse/YARN-3284
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
Attachments: YARN-3284.1.patch, YARN-3284.1.patch, YARN-3284.2.patch,
YARN-3284.3.patch
Current, we have some extra metrics about the application and current attempt
in RM Web UI. We should expose that information through YARN Command, too.
1. Preemption metrics
2. application outstanding resource requests
3. container locality info
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361728#comment-14361728 ] Hudson commented on YARN-2854: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #132 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/132/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md * hadoop-yarn-project/CHANGES.txt The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361726#comment-14361726 ] Hudson commented on YARN-3267: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #132 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/132/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/CHANGES.txt Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2854) The document about timeline service and generic service needs to be updated
[
https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-2854:
Attachment: YARN-2854.20150314-1_branch2.patch
YARN-2854.20150314-1.patch
Hi [~zjshen],
There were some difference in branch 2 and trunk when it was converted from apt
to markdown(yarn-3285) hence it failed. Also i felt those modification was
required {{$HADOOP_YARN_HOME/sbin/yarn-daemon.sh start timelineserver }} hence
have modified the patch for the trunk too.. Apart from this i had missed some
small modification regarding the {{current status}} from the previous patch
(YARN-2854.20150311-1.patch) . Have attached patches incorporating these
corrections for both trunk and branch-2. Sorry for the inconvenience.
The document about timeline service and generic service needs to be updated
---
Key: YARN-2854
URL: https://issues.apache.org/jira/browse/YARN-2854
Project: Hadoop YARN
Issue Type: Improvement
Components: timelineserver
Reporter: Zhijie Shen
Assignee: Naganarasimha G R
Priority: Critical
Attachments: TimelineServer.html, YARN-2854.20141120-1.patch,
YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch,
YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch,
YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch,
timeline_structure.jpg
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361749#comment-14361749 ] Hudson commented on YARN-2854: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #866 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/866/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361747#comment-14361747 ] Hudson commented on YARN-3267: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #866 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/866/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[
https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361796#comment-14361796
]
Hadoop QA commented on YARN-2854:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704591/YARN-2854.20150314-1_branch2.patch
against trunk revision bd0a9ba.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6961//console
This message is automatically generated.
The document about timeline service and generic service needs to be updated
---
Key: YARN-2854
URL: https://issues.apache.org/jira/browse/YARN-2854
Project: Hadoop YARN
Issue Type: Improvement
Components: timelineserver
Reporter: Zhijie Shen
Assignee: Naganarasimha G R
Priority: Critical
Attachments: TimelineServer.html, YARN-2854.20141120-1.patch,
YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch,
YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch,
YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch,
timeline_structure.jpg
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361835#comment-14361835 ] Hudson commented on YARN-2854: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #132 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/132/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361822#comment-14361822 ] Hudson commented on YARN-3267: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #123 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/123/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-yarn-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361824#comment-14361824 ] Hudson commented on YARN-2854: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #123 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/123/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md * hadoop-yarn-project/CHANGES.txt The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361833#comment-14361833 ] Hudson commented on YARN-3267: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #132 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/132/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-yarn-project/CHANGES.txt Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361854#comment-14361854 ] Hudson commented on YARN-3267: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2082 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2082/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-yarn-project/CHANGES.txt Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361856#comment-14361856 ] Hudson commented on YARN-2854: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2082 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2082/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3339) TestDockerContainerExecutor should pull a single image and not the entire centos repository
[ https://issues.apache.org/jira/browse/YARN-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361869#comment-14361869 ] Ravindra Kumar Naik commented on YARN-3339: --- [~vvasudev] [~airbots] [~raviprak] its a small fix, could anyone please review it? TestDockerContainerExecutor should pull a single image and not the entire centos repository --- Key: YARN-3339 URL: https://issues.apache.org/jira/browse/YARN-3339 Project: Hadoop YARN Issue Type: Test Components: test Affects Versions: 2.6.0 Environment: Linux Reporter: Ravindra Kumar Naik Priority: Minor Fix For: 2.6.0 Attachments: YARN-3339-branch-2.6.0.001.patch, YARN-3339-trunk.001.patch TestDockerContainerExecutor test pulls the entire centos repository which is time consuming. Pulling a specific image (e.g. centos7) will be sufficient to run the test successfully and will save time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3302) TestDockerContainerExecutor should run automatically if it can detect docker in the usual place
[ https://issues.apache.org/jira/browse/YARN-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361870#comment-14361870 ] Ravindra Kumar Naik commented on YARN-3302: --- [~vvasudev] [~airbots] [~raviprak] could anyone please review the patch? TestDockerContainerExecutor should run automatically if it can detect docker in the usual place --- Key: YARN-3302 URL: https://issues.apache.org/jira/browse/YARN-3302 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.6.0 Reporter: Ravi Prakash Attachments: YARN-3302-trunk.001.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[ https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361810#comment-14361810 ] Hudson commented on YARN-2854: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2064 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2064/]) YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. (zjshen: rev 6fdef76cc3e818856ddcc4d385c2899a8e6ba916) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/timeline_structure.jpg * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md The document about timeline service and generic service needs to be updated --- Key: YARN-2854 URL: https://issues.apache.org/jira/browse/YARN-2854 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: Zhijie Shen Assignee: Naganarasimha G R Priority: Critical Attachments: TimelineServer.html, YARN-2854.20141120-1.patch, YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch, YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch, YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch, timeline_structure.jpg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3267) Timelineserver applies the ACL rules after applying the limit on the number of records
[ https://issues.apache.org/jira/browse/YARN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361808#comment-14361808 ] Hudson commented on YARN-3267: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2064 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2064/]) YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) (jeagles: rev 8180e676abb2bb500a48b3a0c0809d2a807ab235) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TestTimelineDataManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineReader.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java Timelineserver applies the ACL rules after applying the limit on the number of records -- Key: YARN-3267 URL: https://issues.apache.org/jira/browse/YARN-3267 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.6.0 Reporter: Prakash Ramachandran Assignee: Chang Li Fix For: 2.7.0 Attachments: YARN-3267.3.patch, YARN-3267.4.patch, YARN-3267.5.patch, YARN_3267_V1.patch, YARN_3267_V2.patch, YARN_3267_WIP.patch, YARN_3267_WIP1.patch, YARN_3267_WIP2.patch, YARN_3267_WIP3.patch While fetching the entities from timelineserver, the limit is applied on the entities to be fetched from leveldb, the ACL filters are applied after this (TimelineDataManager.java::getEntities). this could mean that even if there are entities available which match the query criteria, we could end up not getting any results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
