[jira] [Updated] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1335: - Attachment: YARN-1335-1.patch Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803891#comment-13803891 ] Sandy Ryza commented on YARN-1335: -- Thanks for the review Tsuyoshi. Uploading a patch

[jira] [Updated] (YARN-975) Add a file-system implementation for history-storage

[ https://issues.apache.org/jira/browse/YARN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-975: - Attachment: YARN-975.10.patch Update the patch according the change in YARN-947 and YARN-956 Add a

[jira] [Commented] (YARN-975) Add a file-system implementation for history-storage

[ https://issues.apache.org/jira/browse/YARN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803902#comment-13803902 ] Hadoop QA commented on YARN-975: {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (YARN-1344) Separate ApplicationAttemptStartDataProto and ApplicationAttemptRegisteredDataProto

[ https://issues.apache.org/jira/browse/YARN-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1344: -- Issue Type: Sub-task (was: Improvement) Parent: YARN-321 Separate

[jira] [Created] (YARN-1344) Separate ApplicationAttemptStartDataProto and ApplicationAttemptRegisteredDataProto

Zhijie Shen created YARN-1344: - Summary: Separate ApplicationAttemptStartDataProto and ApplicationAttemptRegisteredDataProto Key: YARN-1344 URL: https://issues.apache.org/jira/browse/YARN-1344 Project:

[jira] [Commented] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803906#comment-13803906 ] Jian He commented on YARN-891: -- The TestZKRMStateStoreZKClientConnections failure is unrelated.

[jira] [Created] (YARN-1345) RMServerUtils#createApplicationAttemptState needs to add the mapping for FINAL_SAVING

Zhijie Shen created YARN-1345: - Summary: RMServerUtils#createApplicationAttemptState needs to add the mapping for FINAL_SAVING Key: YARN-1345 URL: https://issues.apache.org/jira/browse/YARN-1345 Project:

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803915#comment-13803915 ] Hadoop QA commented on YARN-1335: - {color:red}-1 overall{color}. Here are the results of

[jira] [Created] (YARN-1346) Revisit the output type of the reader interface

Zhijie Shen created YARN-1346: - Summary: Revisit the output type of the reader interface Key: YARN-1346 URL: https://issues.apache.org/jira/browse/YARN-1346 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803931#comment-13803931 ] Bikas Saha commented on YARN-891: - [~jianhe] Can we please have some comments that describe

[jira] [Commented] (YARN-1321) NMTokenCache is a a singleton, prevents multiple AMs running in a single JVM to work correctly

[ https://issues.apache.org/jira/browse/YARN-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803936#comment-13803936 ] Sandy Ryza commented on YARN-1321: -- [~vinodkv], if I understand correctly, the difference

[jira] [Commented] (YARN-895) RM crashes if it restarts while NameNode is in safe mode

[ https://issues.apache.org/jira/browse/YARN-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803945#comment-13803945 ] Bikas Saha commented on YARN-895: - This does not look like a complete fix since this can

[jira] [Commented] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803984#comment-13803984 ] Bikas Saha commented on YARN-891: - Is APP_SAVED event being overloaded for both saving

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804038#comment-13804038 ] Luke Lu commented on YARN-311: -- v9 patch lgtm. +1. For the record, we've discussed the changes

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804088#comment-13804088 ] Tsuyoshi OZAWA commented on YARN-1307: -- {quote} When the data size exceeds the size of

[jira] [Commented] (YARN-1343) NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs

[ https://issues.apache.org/jira/browse/YARN-1343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804200#comment-13804200 ] Alejandro Abdelnur commented on YARN-1343: -- [~bikassaha], I disagree that this is

[jira] [Commented] (YARN-1321) NMTokenCache is a a singleton, prevents multiple AMs running in a single JVM to work correctly

[ https://issues.apache.org/jira/browse/YARN-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804206#comment-13804206 ] Alejandro Abdelnur commented on YARN-1321: -- [~vinodkv], kind of agree with

[jira] [Assigned] (YARN-1337) Recover active container state upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash reassigned YARN-1337: -- Assignee: Ravi Prakash Recover active container state upon nodemanager restart

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804320#comment-13804320 ] Luke Lu commented on YARN-311: -- [~djp]: Upon second thought, I think we should improve the

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804335#comment-13804335 ] Luke Lu commented on YARN-311: -- We can probably infer the option by an overcommit timeout

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804382#comment-13804382 ] Bikas Saha commented on YARN-1307: -- Not quite sure what you mean by encoding. There is

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804383#comment-13804383 ] Bikas Saha commented on YARN-1307: -- btw, for the purpose of this jira we can postpone the

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804465#comment-13804465 ] Junping Du commented on YARN-311: - Thanks Luke for review and comments! Yes. We discussed

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804532#comment-13804532 ] Luke Lu commented on YARN-311: -- bq. Other policies/options will be addressed in YARN-999

[jira] [Commented] (YARN-1068) Add admin support for HA operations

[ https://issues.apache.org/jira/browse/YARN-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804583#comment-13804583 ] Karthik Kambatla commented on YARN-1068: Tested the latest patch again on a real

[jira] [Updated] (YARN-1181) Augment MiniYARNCluster to support HA mode

[ https://issues.apache.org/jira/browse/YARN-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1181: --- Summary: Augment MiniYARNCluster to support HA mode (was: Implement MiniYARNHACluster)

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804690#comment-13804690 ] Junping Du commented on YARN-311: - Luke, instead of void setTotalCapacity(Resource resource,

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804697#comment-13804697 ] Alejandro Abdelnur commented on YARN-1335: -- * SchedulerApplication ** public

[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes

[ https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804719#comment-13804719 ] Hadoop QA commented on YARN-311: {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804718#comment-13804718 ] Sandy Ryza commented on YARN-1335: -- Looked into this and I don't think synchronization

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804743#comment-13804743 ] Alejandro Abdelnur commented on YARN-1335: -- thanks for clarifying, +1 then. Move

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804752#comment-13804752 ] Tsuyoshi OZAWA commented on YARN-1307: -- {code} Not quite sure what you mean by

[jira] [Created] (YARN-1348) Batching optimization for ZKRMStateStore

Tsuyoshi OZAWA created YARN-1348: Summary: Batching optimization for ZKRMStateStore Key: YARN-1348 URL: https://issues.apache.org/jira/browse/YARN-1348 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804777#comment-13804777 ] Bikas Saha commented on YARN-1307: -- [~jianhe] Please correct me if I am wrong. The

[jira] [Moved] (YARN-1349) yarn.cmd does not support passthrough to any arbitrary class.

[ https://issues.apache.org/jira/browse/YARN-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth moved HDFS-5414 to YARN-1349: --- Component/s: (was: scripts) client

[jira] [Commented] (YARN-978) [YARN-321] Adding ApplicationAttemptReport and Protobuf implementation

[ https://issues.apache.org/jira/browse/YARN-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804788#comment-13804788 ] Mayank Bansal commented on YARN-978: Assigning it back to myself for change due to

[jira] [Assigned] (YARN-978) [YARN-321] Adding ApplicationAttemptReport and Protobuf implementation

[ https://issues.apache.org/jira/browse/YARN-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal reassigned YARN-978: -- Assignee: Mayank Bansal (was: Xuan Gong) [YARN-321] Adding ApplicationAttemptReport and

[jira] [Updated] (YARN-674) Slow or failing DelegationToken renewals on submission itself make RM unavailable

[ https://issues.apache.org/jira/browse/YARN-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omkar Vinit Joshi updated YARN-674: --- Attachment: YARN-674.1.patch Slow or failing DelegationToken renewals on submission itself

[jira] [Created] (YARN-1350) Should not add Lost Node by NodeManager reboot

Shinichi Yamashita created YARN-1350: Summary: Should not add Lost Node by NodeManager reboot Key: YARN-1350 URL: https://issues.apache.org/jira/browse/YARN-1350 Project: Hadoop YARN

[jira] [Commented] (YARN-1335) Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication

[ https://issues.apache.org/jira/browse/YARN-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804803#comment-13804803 ] Hudson commented on YARN-1335: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4654 (See

[jira] [Updated] (YARN-786) Expose application resource usage in RM REST API

[ https://issues.apache.org/jira/browse/YARN-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-786: Attachment: YARN-786.patch Expose application resource usage in RM REST API

[jira] [Commented] (YARN-1336) Work-preserving nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804820#comment-13804820 ] Joep Rottinghuis commented on YARN-1336: We're finding the same during our cluster

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804823#comment-13804823 ] Tsuyoshi OZAWA commented on YARN-1307: -- IIUC, You're correct, Bikas. What I'd like to

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804830#comment-13804830 ] Bikas Saha commented on YARN-1307: -- Encryption is probably not required. We depend on ZK

[jira] [Commented] (YARN-674) Slow or failing DelegationToken renewals on submission itself make RM unavailable

[ https://issues.apache.org/jira/browse/YARN-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804834#comment-13804834 ] Omkar Vinit Joshi commented on YARN-674: updating the patch Modifying the event flow

[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA

[ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804837#comment-13804837 ] Tsuyoshi OZAWA commented on YARN-1307: -- OK. This is the my thought of znode structure.

[jira] [Commented] (YARN-786) Expose application resource usage in RM REST API

[ https://issues.apache.org/jira/browse/YARN-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804844#comment-13804844 ] Sandy Ryza commented on YARN-786: - Attached patch adds allocatedMB, allocatedVCores, and

[jira] [Commented] (YARN-956) [YARN-321] Add a testable in-memory HistoryStorage

[ https://issues.apache.org/jira/browse/YARN-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804853#comment-13804853 ] Vinod Kumar Vavilapalli commented on YARN-956: -- Seems like the following will

[jira] [Created] (YARN-1351) Invalid Format String Invocation

Konstantin Weitz created YARN-1351: -- Summary: Invalid Format String Invocation Key: YARN-1351 URL: https://issues.apache.org/jira/browse/YARN-1351 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-1351) Invalid string format in Fair Scheduler log warn message

[ https://issues.apache.org/jira/browse/YARN-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1351: - Summary: Invalid string format in Fair Scheduler log warn message (was: Invalid Format String

[jira] [Commented] (YARN-1351) Invalid string format in Fair Scheduler log warn message

[ https://issues.apache.org/jira/browse/YARN-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804865#comment-13804865 ] Sandy Ryza commented on YARN-1351: -- Thanks for finding and working on this [~konnew88]!

[jira] [Updated] (YARN-1181) Augment MiniYARNCluster to support HA mode

[ https://issues.apache.org/jira/browse/YARN-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1181: --- Attachment: yarn-1181-2.patch Rebased patch on trunk. This applies on top of YARN-1068.

[jira] [Updated] (YARN-956) [YARN-321] Add a testable in-memory HistoryStorage

[ https://issues.apache.org/jira/browse/YARN-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-956: - Attachment: YARN-956.8.patch Updated the patch according Vinod's latest comments. [YARN-321] Add a

[jira] [Updated] (YARN-1181) Augment MiniYARNCluster to support HA mode

[ https://issues.apache.org/jira/browse/YARN-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1181: --- Attachment: yarn-1181-2.patch Augment MiniYARNCluster to support HA mode

[jira] [Updated] (YARN-1181) Augment MiniYARNCluster to support HA mode

[ https://issues.apache.org/jira/browse/YARN-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1181: --- Attachment: (was: yarn-1181-2.patch) Augment MiniYARNCluster to support HA mode

[jira] [Updated] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-891: - Attachment: YARN-891.7.patch bq. Is APP_SAVED event being overloaded for both saving initial and final state?

[jira] [Commented] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804905#comment-13804905 ] Jian He commented on YARN-891: -- Summarize the patch: - RMStateStore change: -- Add more

[jira] [Updated] (YARN-891) Store completed application information in RM state store

[ https://issues.apache.org/jira/browse/YARN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-891: - Attachment: YARN-891.7.patch Store completed application information in RM state store

[jira] [Updated] (YARN-1172) Convert *SecretManagers in the RM to services

[ https://issues.apache.org/jira/browse/YARN-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-1172: - Attachment: YARN-1172.8.patch Rebased on trunk and fixed to use --no-prefix option to generate a