[jira] [Updated] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-04 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6820: - Attachment: YARN-6820-YARN-5355.002.patch Attaching patch 002 , updated as per review recommendations.

[jira] [Updated] (YARN-6947) The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6947: -- Description: Each time the FairScheduler assign container, it will checks whether the resources used by

[jira] [Resolved] (YARN-6947) The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu resolved YARN-6947. Resolution: Duplicate > The implementation of Schedulable#getResourceUsage so inefficiency that can >

[jira] [Comment Edited] (YARN-6788) Improve performance of resource profile branch

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114043#comment-16114043 ] Sunil G edited comment on YARN-6788 at 8/4/17 8:13 AM: --- Thanks [~templedf] Quick

[jira] [Commented] (YARN-6920) Fix TestNMClient failure due to YARN-6706

2017-08-04 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114006#comment-16114006 ] Jian He commented on YARN-6920: --- I see, thanks for the explanation, so, it's guaranteed the Guaranteed

[jira] [Commented] (YARN-6920) Fix TestNMClient failure due to YARN-6706

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114021#comment-16114021 ] Arun Suresh commented on YARN-6920: --- True.. the sequence of events you mentioned can happen. I am hoping

[jira] [Created] (YARN-6947) The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling

2017-08-04 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-6947: - Summary: The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling Key: YARN-6947 URL:

[jira] [Commented] (YARN-6788) Improve performance of resource profile branch

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114043#comment-16114043 ] Sunil G commented on YARN-6788: --- Thanks [~templedf] Quick clarification for few points: bq.the findbugs

[jira] [Commented] (YARN-6873) Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice

2017-08-04 Thread Wenxin He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113991#comment-16113991 ] Wenxin He commented on YARN-6873: - [~Cyl], since HADOOP-14706 is commited, would you use the helper method

[jira] [Updated] (YARN-6938) Add a flag to indicate whether timeline server acl enabled

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6938: -- Component/s: timelineserver > Add a flag to indicate whether timeline server acl enabled >

[jira] [Updated] (YARN-6947) The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6947: -- Description: {code:java} @Override public Resource assignContainer(FSSchedulerNode node) {

[jira] [Comment Edited] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-04 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114028#comment-16114028 ] Vrushali C edited comment on YARN-6820 at 8/4/17 7:07 AM: -- Attaching patch 002 ,

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114094#comment-16114094 ] Yufei Gu commented on YARN-6361: You can take it. > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114061#comment-16114061 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] Hi, Yufei. I would like to work on this JIRA if you have

[jira] [Updated] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated YARN-6951: Description: {code:title=LinuxContainerExecutor.java} ... ... if (LOG.isDebugEnabled()) {

[jira] [Assigned] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang reassigned YARN-6951: --- Assignee: Yang Wang > Fix debug log when Resource handler chain is enabled >

[jira] [Commented] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-04 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114235#comment-16114235 ] Rohith Sharma K S commented on YARN-6133: - thanks @varun for the patch. Some comments # Token is

[jira] [Created] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Yang Wang (JIRA)
Yang Wang created YARN-6951: --- Summary: Fix debug log when Resource handler chain is enabled Key: YARN-6951 URL: https://issues.apache.org/jira/browse/YARN-6951 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated YARN-6951: Attachment: YARN-6951.001.patch > Fix debug log when Resource handler chain is enabled >

[jira] [Commented] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-04 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114248#comment-16114248 ] Varun Saxena commented on YARN-6133: bq. Token is renewed just before 10 seconds. Should it be

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114258#comment-16114258 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] Thank Yufei. For this question, including the

[jira] [Assigned] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou reassigned YARN-6361: - Assignee: YunFan Zhou (was: Yufei Gu) > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage

[jira] [Created] (YARN-6948) Invalid event: ATTEMPT_ADDED at FINAL_SAVING

2017-08-04 Thread lujie (JIRA)
lujie created YARN-6948: --- Summary: Invalid event: ATTEMPT_ADDED at FINAL_SAVING Key: YARN-6948 URL: https://issues.apache.org/jira/browse/YARN-6948 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-6950) Invalid event: LAUNCH_FAILED at FAILED

2017-08-04 Thread lujie (JIRA)
lujie created YARN-6950: --- Summary: Invalid event: LAUNCH_FAILED at FAILED Key: YARN-6950 URL: https://issues.apache.org/jira/browse/YARN-6950 Project: Hadoop YARN Issue Type: Bug Components:

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2017-08-04 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114150#comment-16114150 ] Yang Wang commented on YARN-5621: - {code:title=LinuxContainerExecutor.java} protected void

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2017-08-04 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114150#comment-16114150 ] Yang Wang edited comment on YARN-5621 at 8/4/17 9:08 AM: -

[jira] [Created] (YARN-6949) Invalid event: LOCALIZED at LOCALIZED

2017-08-04 Thread lujie (JIRA)
lujie created YARN-6949: --- Summary: Invalid event: LOCALIZED at LOCALIZED Key: YARN-6949 URL: https://issues.apache.org/jira/browse/YARN-6949 Project: Hadoop YARN Issue Type: Bug Components:

[jira] [Commented] (YARN-6726) Fix issues with docker commands executed by container-executor

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114160#comment-16114160 ] Sunil G commented on YARN-6726: --- Sorry for pitching in late here: Some minor comments: # It will be better

[jira] [Commented] (YARN-6788) Improve performance of resource profile branch

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114705#comment-16114705 ] Sunil G commented on YARN-6788: --- Thank you very much for thorough reviews and commit [~templedf] and

[jira] [Commented] (YARN-6871) Add additional deSelects params in getAppReport

2017-08-04 Thread Giovanni Matteo Fumarola (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114710#comment-16114710 ] Giovanni Matteo Fumarola commented on YARN-6871: Thanks [~tanujnay] for the patch. Few

[jira] [Updated] (YARN-3254) HealthReport should include disk full information

2017-08-04 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated YARN-3254: --- Attachment: YARN-3254-005.patch > HealthReport should include disk full information >

[jira] [Comment Edited] (YARN-3254) HealthReport should include disk full information

2017-08-04 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114935#comment-16114935 ] Suma Shivaprasad edited comment on YARN-3254 at 8/4/17 8:30 PM: Updated the

[jira] [Updated] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-6955: --- Attachment: YARN-6955.v1.patch > Concurrent registerAM thread in Federation Interceptor >

[jira] [Commented] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-04 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114990#comment-16114990 ] Jason Lowe commented on YARN-6820: -- Thanks for updating the patch! The javadoc errors are relevant:

[jira] [Resolved] (YARN-6944) The comment about ResourceManager#createPolicyMonitors lies

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu resolved YARN-6944. Resolution: Duplicate > The comment about ResourceManager#createPolicyMonitors lies >

[jira] [Commented] (YARN-6952) Enable scheduling monitor in FS

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114904#comment-16114904 ] Yufei Gu commented on YARN-6952: Uploaded patch v1. File YARN-6954 to remove interface

[jira] [Created] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Botong Huang (JIRA)
Botong Huang created YARN-6955: -- Summary: Concurrent registerAM thread in Federation Interceptor Key: YARN-6955 URL: https://issues.apache.org/jira/browse/YARN-6955 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6033: - Attachment: YARN-6033.009.patch Attached ver.009 patch, fixed warnings. > Add support for sections in

[jira] [Updated] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-6955: --- Attachment: (was: YARN-6955.v1.patch) > Concurrent registerAM thread in Federation Interceptor >

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114654#comment-16114654 ] Yufei Gu commented on YARN-6361: Thanks for taking this, [~daemon]. This jira is dedicated to performance

[jira] [Comment Edited] (YARN-6789) new api to get all supported resources from RM

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114722#comment-16114722 ] Sunil G edited comment on YARN-6789 at 8/4/17 5:58 PM: --- Submitting patch to run

[jira] [Commented] (YARN-6726) Fix issues with docker commands executed by container-executor

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114746#comment-16114746 ] Hadoop QA commented on YARN-6726: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Resolved] (YARN-6934) ResourceUtils.checkMandatoryResources() should also ensure that no min or max is set for vcores or memory

2017-08-04 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved YARN-6934. Resolution: Invalid > ResourceUtils.checkMandatoryResources() should also ensure that no min

[jira] [Commented] (YARN-6934) ResourceUtils.checkMandatoryResources() should also ensure that no min or max is set for vcores or memory

2017-08-04 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114863#comment-16114863 ] Daniel Templeton commented on YARN-6934: Thanks for the patch, [~maniraj...@gmail.com]. I just

[jira] [Updated] (YARN-4090) Make Collections.sort() more efficient by caching resource usage

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-4090: --- Summary: Make Collections.sort() more efficient by caching resource usage (was: Make Collections.sort() more

[jira] [Updated] (YARN-6726) Fix issues with docker commands executed by container-executor

2017-08-04 Thread Shane Kumpf (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Kumpf updated YARN-6726: -- Attachment: YARN-6726.003.patch > Fix issues with docker commands executed by container-executor >

[jira] [Commented] (YARN-6726) Fix issues with docker commands executed by container-executor

2017-08-04 Thread Shane Kumpf (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114829#comment-16114829 ] Shane Kumpf commented on YARN-6726: --- Thanks for the review, [~sunilg]. I have attached a new patch that

[jira] [Updated] (YARN-4090) Make Collections.sort() more efficient by

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-4090: --- Summary: Make Collections.sort() more efficient by (was: Make Collections.sort() more efficient in

[jira] [Commented] (YARN-6892) Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114721#comment-16114721 ] Sunil G commented on YARN-6892: --- YARN-6788 is committed. Submitting patch for jenkins. > Improve API

[jira] [Created] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-04 Thread Daniel Templeton (JIRA)
Daniel Templeton created YARN-6953: -- Summary: Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() Key: YARN-6953 URL:

[jira] [Updated] (YARN-6952) Enable scheduling monitor in FS

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6952: --- Attachment: YARN-6952.001.patch > Enable scheduling monitor in FS > --- > >

[jira] [Updated] (YARN-6934) ResourceUtils.checkMandatoryResources() should also ensure that no min or max is set for vcores or memory

2017-08-04 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-6934: --- Attachment: YARN-6934.001.patch > ResourceUtils.checkMandatoryResources() should also ensure that no

[jira] [Commented] (YARN-65) Reduce RM app memory footprint once app has completed

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114793#comment-16114793 ] Hadoop QA commented on YARN-65: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime

[jira] [Created] (YARN-6952) Enable scheduling monitor in FS

2017-08-04 Thread Yufei Gu (JIRA)
Yufei Gu created YARN-6952: -- Summary: Enable scheduling monitor in FS Key: YARN-6952 URL: https://issues.apache.org/jira/browse/YARN-6952 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-6934) ResourceUtils.checkMandatoryResources() should also ensure that no min or max is set for vcores or memory

2017-08-04 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114849#comment-16114849 ] Manikandan R commented on YARN-6934: Based on discussion, Attached patch for review. It contains even

[jira] [Created] (YARN-6954) Remove interface PreemptableResourceScheduler

2017-08-04 Thread Yufei Gu (JIRA)
Yufei Gu created YARN-6954: -- Summary: Remove interface PreemptableResourceScheduler Key: YARN-6954 URL: https://issues.apache.org/jira/browse/YARN-6954 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-3254) HealthReport should include disk full information

2017-08-04 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114935#comment-16114935 ] Suma Shivaprasad commented on YARN-3254: Updated the patch to display the exact root cause for the

[jira] [Updated] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-6955: --- Attachment: YARN-6955.v1.patch > Concurrent registerAM thread in Federation Interceptor >

[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK

2017-08-04 Thread Ellen Hui (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0003-Registry-API-api-only.patch This patch only adds the API, record, types, and exceptions,

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115205#comment-16115205 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] Thank Yufei. Sorry, I'm off the subject. But either way,

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115205#comment-16115205 ] YunFan Zhou edited comment on YARN-6361 at 8/5/17 1:49 AM: --- [~yufeigu] Thank

[jira] [Updated] (YARN-5938) Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5938: -- Fix Version/s: 2.9.0 > Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey > instead

[jira] [Commented] (YARN-6952) Enable scheduling monitor in FS

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115156#comment-16115156 ] Hadoop QA commented on YARN-6952: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-6802) Add Max AM Resource and AM Resource Usage to Leaf Queue View in FairScheduler WebUI

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6802: --- Attachment: YARN-6802.branch-2.001.patch > Add Max AM Resource and AM Resource Usage to Leaf Queue View in

[jira] [Commented] (YARN-65) Reduce RM app memory footprint once app has completed

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115160#comment-16115160 ] Hadoop QA commented on YARN-65: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime

[jira] [Commented] (YARN-6945) Display the ACL of leaf queue in RM scheduler page

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115185#comment-16115185 ] YunFan Zhou commented on YARN-6945: --- [~Naganarasimha] Hi, Naganarasimha G R. Any suggestion? > Display

[jira] [Commented] (YARN-6952) Enable scheduling monitor in FS

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115162#comment-16115162 ] Yufei Gu commented on YARN-6952: The test failure is unrelated. > Enable scheduling monitor in FS >

[jira] [Commented] (YARN-6802) Add Max AM Resource and AM Resource Usage to Leaf Queue View in FairScheduler WebUI

2017-08-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115161#comment-16115161 ] Yufei Gu commented on YARN-6802: Uploaded the patch for branch-2 and committed it to branch-2. > Add Max

[jira] [Commented] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Subru Krishnan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115167#comment-16115167 ] Subru Krishnan commented on YARN-6955: -- Thanks [~botong] for surfacing this issue. The patch looks

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-04 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115181#comment-16115181 ] Miklos Szegedi commented on YARN-6033: -- Sorry, I think I found one more in the latest patch. {code} 58

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Priority: Major (was: Minor) > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115205#comment-16115205 ] YunFan Zhou edited comment on YARN-6361 at 8/5/17 1:49 AM: --- [~yufeigu] Thank

[jira] [Commented] (YARN-6949) Invalid event: LOCALIZATION_FAILED at LOCALIZED

2017-08-04 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115220#comment-16115220 ] lujie commented on YARN-6949: - I check the log and also find some NullPointerException: {code:java}

[jira] [Updated] (YARN-5966) AMRMClient changes to support ExecutionType update

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5966: -- Fix Version/s: 2.9.0 > AMRMClient changes to support ExecutionType update >

[jira] [Commented] (YARN-5966) AMRMClient changes to support ExecutionType update

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115085#comment-16115085 ] Arun Suresh commented on YARN-5966: --- Committed this to branch-2 > AMRMClient changes to support

[jira] [Commented] (YARN-6811) [ATS1.5] All history logs should be kept under its own User Directory.

2017-08-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115124#comment-16115124 ] Junping Du commented on YARN-6811: -- I have commit the patch to trunk. For branch-2, my cherry-pick has

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Fix Version/s: 2.9.0 > Support for ApplicationMasterService processing chain of interceptors >

[jira] [Commented] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115127#comment-16115127 ] Arun Suresh commented on YARN-6777: --- Committed to branch-2 > Support for ApplicationMasterService

[jira] [Updated] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6852: - Attachment: (was: YARN-6033.009.patch) > [YARN-6223] Native code changes to support isolate GPU

[jira] [Updated] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6852: - Attachment: YARN-6033.009.patch > [YARN-6223] Native code changes to support isolate GPU devices by using

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115055#comment-16115055 ] Wangda Tan commented on YARN-6852: -- Hi Miklos, really appreciate your thorough reviews, very helpful! I

[jira] [Updated] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6852: - Attachment: YARN-6852.003.patch > [YARN-6223] Native code changes to support isolate GPU devices by using

[jira] [Comment Edited] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-04 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115055#comment-16115055 ] Wangda Tan edited comment on YARN-6852 at 8/4/17 10:46 PM: --- Hi Miklos, really

[jira] [Commented] (YARN-6811) [ATS1.5] All history logs should be kept under its own User Directory.

2017-08-04 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115099#comment-16115099 ] Hudson commented on YARN-6811: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12122 (See

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115111#comment-16115111 ] Hadoop QA commented on YARN-6033: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-3254) HealthReport should include disk full information

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115112#comment-16115112 ] Hadoop QA commented on YARN-3254: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115117#comment-16115117 ] Arun Suresh commented on YARN-6776: --- Committed this to branch-2 > Refactor ApplicaitonMasterService to

[jira] [Updated] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-08-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6776: -- Fix Version/s: 2.9.0 > Refactor ApplicaitonMasterService to move actual processing logic to a >

[jira] [Commented] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115131#comment-16115131 ] Hadoop QA commented on YARN-6955: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-3254) HealthReport should include disk full information

2017-08-04 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115142#comment-16115142 ] Suma Shivaprasad commented on YARN-3254: [~sunilg] Can you pls review the updated patch? >

[jira] [Commented] (YARN-6634) [API] Refactor ResourceManager WebServices to make API explicit

2017-08-04 Thread Carlo Curino (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115150#comment-16115150 ] Carlo Curino commented on YARN-6634: Thanks [~giovanni.fumarola] for the branch-2 version. Patch looks

[jira] [Updated] (YARN-6955) Concurrent registerAM thread in Federation Interceptor

2017-08-04 Thread Subru Krishnan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subru Krishnan updated YARN-6955: - Issue Type: Sub-task (was: Bug) Parent: YARN-5597 > Concurrent registerAM thread in

[jira] [Assigned] (YARN-3661) Basic Federation UI

2017-08-04 Thread Subru Krishnan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subru Krishnan reassigned YARN-3661: Assignee: Inigo Goiri (was: Giovanni Matteo Fumarola) > Basic Federation UI >

[jira] [Updated] (YARN-6946) Upgrade JUnit from 4 to 5 in hadoop-yarn-common

2017-08-04 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6946: Attachment: YARN-6946.wip001.patch I'm trying to upgrade JUnit 4 to 5 in hadoop-yarn-common. Very

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114258#comment-16114258 ] YunFan Zhou edited comment on YARN-6361 at 8/4/17 11:21 AM: [~yufeigu] Thank

[jira] [Commented] (YARN-6946) Upgrade JUnit from 4 to 5 in hadoop-yarn-common

2017-08-04 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114283#comment-16114283 ] Akira Ajisaka commented on YARN-6946: - I'm thinking it's too early to migrate JUnit from 4 to 5. JUnit

[jira] [Comment Edited] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-04 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114248#comment-16114248 ] Varun Saxena edited comment on YARN-6133 at 8/4/17 12:26 PM: - Thanks

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-04 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114336#comment-16114336 ] Greg Phillips commented on YARN-6930: - Whitelisting runtimes seems to be the best option. I would

[jira] [Updated] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6951: -- Priority: Minor (was: Major) > Fix debug log when Resource handler chain is enabled >

[jira] [Commented] (YARN-6951) Fix debug log when Resource handler chain is enabled

2017-08-04 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114345#comment-16114345 ] Sunil G commented on YARN-6951: --- Looks fine. Will commit once jenkins is run. > Fix debug log when Resource

  1   2   >