[jira] [Commented] (YARN-6885) AllocationFileLoaderService.loadQueue() should use a switch statement in the main tag parsing loop instead of the if/else-if/...

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120150#comment-16120150 ] Hadoop QA commented on YARN-6885: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120200#comment-16120200 ] Manikandan R commented on YARN-6953: [~sunilg] Attached WIP patch to make sure changes are in line with

[jira] [Updated] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-6953: --- Attachment: YARN-6953-YARN-3926-WIP.patch > Clean up

[jira] [Assigned] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-6971: -- Assignee: Yufei Gu > Clean up different ways to create resources >

[jira] [Commented] (YARN-6935) ResourceProfilesManagerImpl.parseResource() has no need of the key parameter

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120347#comment-16120347 ] Hadoop QA commented on YARN-6935: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6033: - Fix Version/s: 3.0.0-beta1 > Add support for sections in container-executor configuration file >

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-09 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120407#comment-16120407 ] Hudson commented on YARN-6033: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12155 (See

[jira] [Commented] (YARN-6903) Yarn-native-service framework core rewrite

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120221#comment-16120221 ] Hadoop QA commented on YARN-6903: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6610) DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120342#comment-16120342 ] Hadoop QA commented on YARN-6610: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-08-09 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120346#comment-16120346 ] Varun Saxena commented on YARN-6323: bq. this is very hard to enforce it from RM. RM can't

[jira] [Commented] (YARN-6917) Queue path is recomputed from scratch on every allocation

2017-08-09 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120417#comment-16120417 ] Jason Lowe commented on YARN-6917: -- Thanks for the patch, Eric! I agree with the checkstyle nit: the new

[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager

2017-08-09 Thread Naganarasimha G R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121046#comment-16121046 ] Naganarasimha G R commented on YARN-6515: - Thanks for the review and commit [~ajisakaa] and reviews

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121050#comment-16121050 ] Jian He commented on YARN-6959: --- ok, the first AM container process exited then, it's impossible for it to

[jira] [Commented] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-09 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121019#comment-16121019 ] Junping Du commented on YARN-6259: -- Thanks for the patch, [~Tao Yang]! It looks like we have performance

[jira] [Updated] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuqi Wang updated YARN-6959: Attachment: YARN-6959.yarn_nm.log.zip Add NM log for this issue. > RM may allocate wrong AM Container for

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121079#comment-16121079 ] Jian He commented on YARN-6959: --- Do you mean step0 is blocked on MARK2 until the this entire process(AM

[jira] [Updated] (YARN-6874) Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite

2017-08-09 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-6874: --- Summary: Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite

[jira] [Commented] (YARN-6874) Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite

2017-08-09 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121077#comment-16121077 ] Varun Saxena commented on YARN-6874: +1. Will commit it shortly. > Supplement timestamp for min

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121095#comment-16121095 ] Yuqi Wang commented on YARN-6959: - I meant heartbeats from Step0 is blocked between MARK1 and MARK3 (i.e.

[jira] [Commented] (YARN-6872) Ensure apps could run given NodeLabels are disabled post RM switchover/restart

2017-08-09 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121027#comment-16121027 ] Junping Du commented on YARN-6872: -- I have backport the commit to branch-2.8.2. > Ensure apps could run

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121060#comment-16121060 ] Yuqi Wang commented on YARN-6959: - [~jianhe] The whole pipeline was that: Step0. AM sent heartbeats to RM.

[jira] [Commented] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121091#comment-16121091 ] Rohith Sharma K S commented on YARN-6133: - committing shortly > [ATSv2 Security] Renew delegation

[jira] [Commented] (YARN-5146) [YARN-3368] Supports Fair Scheduler in new YARN UI

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121096#comment-16121096 ] Sunil G commented on YARN-5146: --- Sorry for last minute hiccup. I earlier tested this patch with a simple

[jira] [Commented] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121104#comment-16121104 ] Rohith Sharma K S commented on YARN-6133: - committed to YARN-5355 branch. since this is depends on

[jira] [Updated] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

2017-08-09 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-5536: -- Priority: Critical (was: Major) Moved the priority to Critical based on the following discussion with [~djp].

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121030#comment-16121030 ] Yuqi Wang commented on YARN-6959: - {code:java} 2017-07-31 21:29:34,047 INFO [Container Monitor]

[jira] [Comment Edited] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121029#comment-16121029 ] Yuqi Wang edited comment on YARN-6959 at 8/10/17 4:12 AM: -- Attach NM log for this

[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update

2017-08-09 Thread kartheek muthyala (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Description: ContainerScheduler should support updateContainer API for - Container Resource

[jira] [Commented] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121065#comment-16121065 ] Yuqi Wang commented on YARN-6959: - Basically, I meant that the allocate RPC call which is sent before AM

[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update

2017-08-09 Thread kartheek muthyala (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Attachment: YARN-5978.001.patch Submitting the initial version of the patch to figure out

[jira] [Commented] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121084#comment-16121084 ] Jian He commented on YARN-6133: --- patch lgtm > [ATSv2 Security] Renew delegation token for app automatically

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121098#comment-16121098 ] Rohith Sharma K S commented on YARN-6323: - Ahh.. I remember the discussion Vrushali. Thanks for

[jira] [Commented] (YARN-5148) [YARN-3368] Add page to new YARN UI to view server side configurations/logs/JVM-metrics

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121097#comment-16121097 ] Sunil G commented on YARN-5148: --- Thanks [~lewuathe], Sorry for delay here. I ll help to review this later

[jira] [Commented] (YARN-6130) [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121103#comment-16121103 ] Rohith Sharma K S commented on YARN-6130: - [~varun_saxena] some of the java docs are failing for

[jira] [Commented] (YARN-6969) Remove method getMinShareMemoryFraction and getPendingContainers in class FairSchedulerQueueInfo

2017-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121054#comment-16121054 ] ASF GitHub Bot commented on YARN-6969: -- GitHub user LarryLo opened a pull request:

[jira] [Comment Edited] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121060#comment-16121060 ] Yuqi Wang edited comment on YARN-6959 at 8/10/17 5:08 AM: -- [~jianhe] The whole

[jira] [Comment Edited] (YARN-6959) RM may allocate wrong AM Container for new attempt

2017-08-09 Thread Yuqi Wang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121060#comment-16121060 ] Yuqi Wang edited comment on YARN-6959 at 8/10/17 5:10 AM: -- [~jianhe] The whole

[jira] [Commented] (YARN-6958) Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119474#comment-16119474 ] Hadoop QA commented on YARN-6958: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119500#comment-16119500 ] Sunil G commented on YARN-6971: --- Thanks [~yufeigu]. bq. at least document when to use

[jira] [Commented] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119505#comment-16119505 ] Yufei Gu commented on YARN-6971: Sounds good to me. Feel free to close this or convert it to subtask. >

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119508#comment-16119508 ] Rohith Sharma K S commented on YARN-6323: - thanks Vrushali for reinitiating this thread. However

[jira] [Commented] (YARN-6975) Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router

2017-08-09 Thread Yeliang Cang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119515#comment-16119515 ] Yeliang Cang commented on YARN-6975: LOG is unused in AppReportFetcher.java, and remove it. > Moving

[jira] [Updated] (YARN-6885) AllocationFileLoaderService.loadQueue() should use a switch statement in the main tag parsing loop instead of the if/else-if/...

2017-08-09 Thread Yu-Tang Lin (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Tang Lin updated YARN-6885: -- Attachment: 0001-YARN-6885.patch > AllocationFileLoaderService.loadQueue() should use a switch statement

[jira] [Commented] (YARN-2098) App priority support in Fair Scheduler

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119468#comment-16119468 ] Sunil G commented on YARN-2098: --- [~ywskycn] General framework for priority support is completed and the CS

[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119471#comment-16119471 ] Sunil G commented on YARN-2099: --- [~ywskycn], please check following

[jira] [Created] (YARN-6975) Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router

2017-08-09 Thread Yeliang Cang (JIRA)
Yeliang Cang created YARN-6975: -- Summary: Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router Key: YARN-6975 URL:

[jira] [Updated] (YARN-6975) Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router

2017-08-09 Thread Yeliang Cang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeliang Cang updated YARN-6975: --- Attachment: YARN-6975.001.patch > Moving logging APIs over to slf4j in hadoop-yarn-server-tests, >

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: For personal environmental reasons, I uploaded an incomplete patch. And I

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: [~yufeigu] [~Naganarasimha] Hi, Yufei, Naganarasimha. I've found that this

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Attachment: (was: YARN-6361.001.pre.patch) > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119528#comment-16119528 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu][~Naganarasimha] Hi, Yufei, Naganarasimha. Could you

[jira] [Commented] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119536#comment-16119536 ] Sunil G commented on YARN-6971: --- Thanks [~yufeigu]. converted as a sub-task > Clean up different ways to

[jira] [Commented] (YARN-6736) Consider writing to both ats v1 & v2 from RM for smoother upgrades

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119478#comment-16119478 ] Rohith Sharma K S commented on YARN-6736: - Thinking on approach, primarily we can decide to publish

[jira] [Commented] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119482#comment-16119482 ] Yufei Gu commented on YARN-6971: Thanks [~sunilg] for the details. I can't think about more to share. In

[jira] [Updated] (YARN-6903) Yarn-native-service framework core rewrite

2017-08-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-6903: -- Attachment: YARN-6903.yarn-native-services.04.patch > Yarn-native-service framework core rewrite >

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: [~yufeigu] Thank you very much, Yufei. I have to say that this JIRA's

[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119495#comment-16119495 ] Sunil G commented on YARN-6953: --- [~maniraj...@gmail.com], I could see that the optimization done here in this

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Attachment: YARN-6361.001.patch > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with

[jira] [Updated] (YARN-6971) Clean up different ways to create resources

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6971: -- Issue Type: Sub-task (was: Improvement) Parent: YARN-3926 > Clean up different ways to create resources

[jira] [Commented] (YARN-6885) AllocationFileLoaderService.loadQueue() should use a switch statement in the main tag parsing loop instead of the if/else-if/...

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119539#comment-16119539 ] Hadoop QA commented on YARN-6885: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119591#comment-16119591 ] Manikandan R edited comment on YARN-6953 at 8/9/17 9:00 AM: [~sunilg] Thanks

[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119591#comment-16119591 ] Manikandan R commented on YARN-6953: [~sunilg] Thanks for review. Agree with your comments. Had

[jira] [Comment Edited] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115846#comment-16115846 ] weiyuan edited comment on YARN-6881 at 8/9/17 9:47 AM: --- Hi [~templedf], I would try

[jira] [Issue Comment Deleted] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiyuan updated YARN-6881: -- Comment: was deleted (was: Hi [~templedf], I would try this issue, could you help for assigning to me. Thanks.

[jira] [Commented] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119644#comment-16119644 ] weiyuan commented on YARN-6881: --- Hi Daniel, I would try this issue, could you help for assigning to me.

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119593#comment-16119593 ] Sunil G commented on YARN-6852: --- Thanks [~leftnoteasy] for the patch. Few minor comments: In

[jira] [Commented] (YARN-6874) TestHBaseStorageFlowRun.testWriteFlowRunMinMax fails intermittently

2017-08-09 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119604#comment-16119604 ] Varun Saxena commented on YARN-6874: [~vrushalic], got a chance to look at the failure. This is coming

[jira] [Updated] (YARN-6133) [ATSv2 Security] Renew delegation token for app automatically if an app collector is active

2017-08-09 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-6133: --- Attachment: YARN-6133-YARN-5355.04.patch Thanks [~jianhe] for the review. Attaching a patch addressing

[jira] [Commented] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119652#comment-16119652 ] Sunil G commented on YARN-6881: --- Added [~v123582] as a contributor and assigned this issue. > LOG is unused

[jira] [Assigned] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G reassigned YARN-6881: - Assignee: weiyuan > LOG is unused in AllocationConfiguration > >

[jira] [Updated] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiyuan updated YARN-6881: -- Attachment: YARN-6881.001.patch > LOG is unused in AllocationConfiguration >

[jira] [Updated] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiyuan updated YARN-6881: -- Attachment: (was: YARN-6881.001.patch) > LOG is unused in AllocationConfiguration >

[jira] [Updated] (YARN-6881) LOG is unused in AllocationConfiguration

2017-08-09 Thread weiyuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiyuan updated YARN-6881: -- Attachment: YARN-6881.001.patch > LOG is unused in AllocationConfiguration >

[jira] [Commented] (YARN-6975) Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119830#comment-16119830 ] Hadoop QA commented on YARN-6975: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-5736) YARN container executor config does not handle white space

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120578#comment-16120578 ] Wangda Tan commented on YARN-5736: -- [~dan...@cloudera.com], [~miklos.szeg...@cloudera.com],

[jira] [Updated] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-09 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6820: - Attachment: YARN-6820-YARN-5355.004.patch Uploading v004. Updates are: - Using empty string "" for

[jira] [Commented] (YARN-6550) Capture launch_container.sh logs

2017-08-09 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120645#comment-16120645 ] Suma Shivaprasad commented on YARN-6550: Yes output is correct. You mean the curl braces between

[jira] [Commented] (YARN-6789) new api to get all supported resources from RM

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120636#comment-16120636 ] Wangda Tan commented on YARN-6789: -- After a offline discussion with [~sunilg], I think we discovered more

[jira] [Updated] (YARN-6874) TestHBaseStorageFlowRun.testWriteFlowRunMinMax fails intermittently

2017-08-09 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6874: - Attachment: YARN-6874-YARN-5355.0001.patch Thanks [~varun_saxena]. yes I think if two writes happen within

[jira] [Commented] (YARN-6550) Capture launch_container.sh logs

2017-08-09 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120571#comment-16120571 ] Allen Wittenauer commented on YARN-6550: are you sure your output is correct? I'm seeing missing

[jira] [Updated] (YARN-6903) Yarn-native-service framework core rewrite

2017-08-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-6903: -- Attachment: YARN-6903.yarn-native-services.05.patch > Yarn-native-service framework core rewrite >

[jira] [Commented] (YARN-6550) Capture launch_container.sh logs

2017-08-09 Thread Suma Shivaprasad (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120554#comment-16120554 ] Suma Shivaprasad commented on YARN-6550: [~aw] Thanks for checking on the bash version portability.

[jira] [Updated] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6852: - Attachment: YARN-6852.005.patch Thanks for comments from [~sunil.gov...@gmail.com], attached 005 patch,

[jira] [Updated] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-09 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6820: - Attachment: YARN-6820-YARN-5355.004.patch > Restrict read access to timelineservice v2 data >

[jira] [Updated] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-09 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6820: - Attachment: (was: YARN-6820-YARN-5355.004.patch) > Restrict read access to timelineservice v2 data >

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-08-09 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120223#comment-16120223 ] Rohith Sharma K S commented on YARN-6323: - bq. we can possibly write entities for running apps only

[jira] [Commented] (YARN-6820) Restrict read access to timelineservice v2 data

2017-08-09 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120278#comment-16120278 ] Jason Lowe commented on YARN-6820: -- bq. DEFAULT_TIMELINE_SERVICE_READ_ALLOWED_USERS should be star *. It

[jira] [Commented] (YARN-6969) Remove method getMinShareMemoryFraction and getPendingContainers in class FairSchedulerQueueInfo

2017-08-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120266#comment-16120266 ] Yufei Gu commented on YARN-6969: Sure. Feel free to take it. Seems like you aren't a contributor.

[jira] [Commented] (YARN-6736) Consider writing to both ats v1 & v2 from RM for smoother upgrades

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120088#comment-16120088 ] Hadoop QA commented on YARN-6736: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6892) Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation

2017-08-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120173#comment-16120173 ] Hadoop QA commented on YARN-6892: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-6610) DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate

2017-08-09 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-6610: --- Attachment: YARN-6610.YARN-3926.002.patch Now that YARN-6788 is in, here's a fresh patch that

[jira] [Commented] (YARN-6935) ResourceProfilesManagerImpl.parseResource() has no need of the key parameter

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120209#comment-16120209 ] Manikandan R commented on YARN-6935: Attached patch for review. >

[jira] [Updated] (YARN-6935) ResourceProfilesManagerImpl.parseResource() has no need of the key parameter

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-6935: --- Attachment: YARN-6935-YARN-3926.001.patch > ResourceProfilesManagerImpl.parseResource() has no need of

[jira] [Assigned] (YARN-6935) ResourceProfilesManagerImpl.parseResource() has no need of the key parameter

2017-08-09 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R reassigned YARN-6935: -- Assignee: Manikandan R > ResourceProfilesManagerImpl.parseResource() has no need of the key

[jira] [Commented] (YARN-5146) [YARN-3368] Supports Fair Scheduler in new YARN UI

2017-08-09 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120265#comment-16120265 ] Sunil G commented on YARN-5146: --- Looks fine. I will commit later today if there are no objections. >

[jira] [Updated] (YARN-6610) DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate

2017-08-09 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-6610: --- Attachment: YARN-6610.YARN-3926.003.patch Realized that I had the sorts backwards. New patch

[jira] [Created] (YARN-6976) Some containers take a long time in KILLING state after the application is finished.

2017-08-09 Thread Aidi Pi (JIRA)
Aidi Pi created YARN-6976: - Summary: Some containers take a long time in KILLING state after the application is finished. Key: YARN-6976 URL: https://issues.apache.org/jira/browse/YARN-6976 Project: Hadoop

[jira] [Created] (YARN-6977) Node information is not provided for non am containers in RM logs

2017-08-09 Thread Sumana Sathish (JIRA)
Sumana Sathish created YARN-6977: Summary: Node information is not provided for non am containers in RM logs Key: YARN-6977 URL: https://issues.apache.org/jira/browse/YARN-6977 Project: Hadoop YARN

[jira] [Updated] (YARN-6977) Node information is not provided for non am containers in RM logs

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6977: - Component/s: capacity scheduler > Node information is not provided for non am containers in RM logs >

[jira] [Updated] (YARN-6977) Node information is not provided for non am containers in RM logs

2017-08-09 Thread Sumana Sathish (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumana Sathish updated YARN-6977: - Description: There is no information on which node non am container is being assigned in the

  1   2   >