[jira] [Updated] (YARN-1874) Cleanup: Move RMActiveServices out of ResourceManager into its own file

[ https://issues.apache.org/jira/browse/YARN-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-1874: - Attachment: YARN-1874.2.patch > Cleanup: Move RMActiveServices out of ResourceManager into its own

[jira] [Updated] (YARN-2022) Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy

[ https://issues.apache.org/jira/browse/YARN-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-2022: -- Description: Cluster Size = 16GB [2NM's] Queue A Capacity = 50% Queue B Capacity = 50% Consider there are 3 appl

[jira] [Created] (YARN-2022) Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy

Sunil G created YARN-2022: - Summary: Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy Key: YARN-2022 UR

[jira] [Assigned] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]

[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G reassigned YARN-2003: - Assignee: Sunil G > Support to process Job priority from Submission Context in > AppAttemptAddedScheduler

[jira] [Commented] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990304#comment-13990304 ] Hadoop QA commented on YARN-1857: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (YARN-2001) Threshold for RM to accept requests from AM after failover

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-2001: -- Description: After failover, RM may require a certain threshold to determine whether it’s safe to make scheduli

[jira] [Assigned] (YARN-1986) After upgrade from 2.2.0 to 2.4.0, NPE on first job start.

[ https://issues.apache.org/jira/browse/YARN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Zhiguo reassigned YARN-1986: - Assignee: Hong Zhiguo > After upgrade from 2.2.0 to 2.4.0, NPE on first job start. > -

[jira] [Updated] (YARN-2001) Threshold for RM to accept requests from AM after failover

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-2001: -- Description: After failover, RM may require a certain threshold to determine whether it’s safe to make schedulin

[jira] [Updated] (YARN-2001) Threshold for RM to accept requests from AM after failover

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-2001: -- Description: RM may not accept allocate requests from AMs until all the NMs have re-synced back with RM. This i

[jira] [Commented] (YARN-2001) Threshold for RM to accept requests from AM after failover

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990254#comment-13990254 ] Jian He commented on YARN-2001: --- bq. Then node1 comes back to RM, RM recovers all containers

[jira] [Updated] (YARN-2001) Threshold for RM to accept requests from AM after failover

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-2001: -- Summary: Threshold for RM to accept requests from AM after failover (was: Persist NMs info for RM restart) > T

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990252#comment-13990252 ] Jian He commented on YARN-2001: --- In a simple case that an application is granted 50% of the c

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990241#comment-13990241 ] Karthik Kambatla commented on YARN-2001: bq. we may run into condition like the res

[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)

[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990239#comment-13990239 ] Subramaniam Krishnan commented on YARN-1708: Attaching the patch > Add a publi

[jira] [Updated] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)

[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subramaniam Krishnan updated YARN-1708: --- Attachment: YARN-1708.patch > Add a public API to reserve resources (part of YARN-1051

[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)

[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990240#comment-13990240 ] Carlo Curino commented on YARN-1708: The attached patch represents a proposal for an e

[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler

[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990233#comment-13990233 ] Wangda Tan commented on YARN-1368: -- Hi [~jianhe], thanks for this patch, I'm agree with ma

[jira] [Commented] (YARN-2010) RM can't transition to active if it can't recover an app attempt

[ https://issues.apache.org/jira/browse/YARN-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990221#comment-13990221 ] Rohith commented on YARN-2010: -- Thank you [~kasha] for reviewing patch. I update the patch for

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990220#comment-13990220 ] Bikas Saha commented on YARN-2001: -- What if users want to have multiple standbys for fault

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990215#comment-13990215 ] Ming Ma commented on YARN-2001: --- 1. In the HA set up, could we make standby RM hot by having

[jira] [Commented] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

[ https://issues.apache.org/jira/browse/YARN-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990210#comment-13990210 ] yeqi commented on YARN-2020: Carlo Curino, thanks for your explaination. You are right, observ

[jira] [Commented] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

[ https://issues.apache.org/jira/browse/YARN-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990203#comment-13990203 ] Carlo Curino commented on YARN-2020: I might be missing your point, but it seems to me

[jira] [Commented] (YARN-2021) Allow AM to set failed final status

[ https://issues.apache.org/jira/browse/YARN-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990196#comment-13990196 ] David Chen commented on YARN-2021: -- I would like to learn more about YARN and would like t

[jira] [Commented] (YARN-1868) YARN status web ui does not show correctly in IE 11

[ https://issues.apache.org/jira/browse/YARN-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990145#comment-13990145 ] Vinod Kumar Vavilapalli commented on YARN-1868: --- Seems reasonable. From what

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990120#comment-13990120 ] Jian He commented on YARN-2001: --- If RM start accepting application requests before NMs sync b

[jira] [Commented] (YARN-2017) Merge common code in schedulers

[ https://issues.apache.org/jira/browse/YARN-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990110#comment-13990110 ] Jian He commented on YARN-2017: --- yup. Put common code in an abstract class from which specif

[jira] [Commented] (YARN-2019) Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990108#comment-13990108 ] Karthik Kambatla commented on YARN-2019: [~djp] - any particular ideas on how this

[jira] [Commented] (YARN-2017) Merge common code in schedulers

[ https://issues.apache.org/jira/browse/YARN-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990104#comment-13990104 ] Karthik Kambatla commented on YARN-2017: s/too/two/ > Merge common code in schedul

[jira] [Commented] (YARN-2017) Merge common code in schedulers

[ https://issues.apache.org/jira/browse/YARN-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990100#comment-13990100 ] Karthik Kambatla commented on YARN-2017: Instead of removing these too, can we make

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990091#comment-13990091 ] Karthik Kambatla commented on YARN-2001: I am not quite sure if this is a good idea

[jira] [Commented] (YARN-1474) Make schedulers services

[ https://issues.apache.org/jira/browse/YARN-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990092#comment-13990092 ] Tsuyoshi OZAWA commented on YARN-1474: -- Yes, it's OK :-) > Make schedulers services >

[jira] [Commented] (YARN-1474) Make schedulers services

[ https://issues.apache.org/jira/browse/YARN-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990088#comment-13990088 ] Karthik Kambatla commented on YARN-1474: Just got back and catching up on a number

[jira] [Commented] (YARN-1987) Wrapper for leveldb DBIterator to aid in handling database exceptions

[ https://issues.apache.org/jira/browse/YARN-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990077#comment-13990077 ] Karthik Kambatla commented on YARN-1987: Looks good to me, except for one nit - Le

[jira] [Created] (YARN-2021) Allow AM to set failed final status

Jakob Homan created YARN-2021: - Summary: Allow AM to set failed final status Key: YARN-2021 URL: https://issues.apache.org/jira/browse/YARN-2021 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-2019) Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990050#comment-13990050 ] Tsuyoshi OZAWA commented on YARN-2019: -- This means that all RM can terminates when ZK

[jira] [Updated] (YARN-2019) Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-2019: - Attachment: YARN-2019.1-wip.patch > Retrospect on decision of making RM crashed if any exception t

[jira] [Commented] (YARN-2019) Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990037#comment-13990037 ] Tsuyoshi OZAWA commented on YARN-2019: -- RMStateStore handles the exceptions in ZKRMSta

[jira] [Commented] (YARN-2010) RM can't transition to active if it can't recover an app attempt

[ https://issues.apache.org/jira/browse/YARN-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990026#comment-13990026 ] Karthik Kambatla commented on YARN-2010: I do think this is a critical issue, but d

[jira] [Updated] (YARN-2010) RM can't transition to active if it can't recover an app attempt

[ https://issues.apache.org/jira/browse/YARN-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2010: --- Priority: Critical (was: Major) Target Version/s: 2.5.0 > RM can't transition to

[jira] [Commented] (YARN-1701) Improve default paths of timeline store and generic history store

[ https://issues.apache.org/jira/browse/YARN-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990009#comment-13990009 ] Tsuyoshi OZAWA commented on YARN-1701: -- Sure. > Improve default paths of timeline sto

[jira] [Commented] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

[ https://issues.apache.org/jira/browse/YARN-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989928#comment-13989928 ] Tsuyoshi OZAWA commented on YARN-2020: -- [~yeqi], thank you for taking this JIRA. Your

[jira] [Commented] (YARN-1906) TestRMRestart#testQueueMetricsOnRMRestart fails intermittently on trunk and branch2

[ https://issues.apache.org/jira/browse/YARN-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989915#comment-13989915 ] Mit Desai commented on YARN-1906: - [~zjshen] and [~wangda], Totally agree. We should add th

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989888#comment-13989888 ] Ashwin Shankar commented on YARN-1864: -- FYI, there is some work going on to fix these

[jira] [Commented] (YARN-1906) TestRMRestart#testQueueMetricsOnRMRestart fails intermittently on trunk and branch2

[ https://issues.apache.org/jira/browse/YARN-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989883#comment-13989883 ] Ashwin Shankar commented on YARN-1906: -- Hey [~mitdesai], I encountered this issue in m

[jira] [Updated] (YARN-1368) Common work to re-populate containers’ state into scheduler

[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-1368: -- Attachment: YARN-1368.1.patch Uploaded a new patch. - AbstractYarnScheduler#recoverContainersOnNode() does the m

[jira] [Updated] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated YARN-1857: -- Attachment: YARN-1857.patch > CapacityScheduler headroom doesn't account for other AM's running > --

[jira] [Commented] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989865#comment-13989865 ] Chen He commented on YARN-1857: --- This failure is related to YARN-1906. > CapacityScheduler h

[jira] [Updated] (YARN-1896) For FairScheduler expose MinimumQueueResource of each queue in QueueMetrics

[ https://issues.apache.org/jira/browse/YARN-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated YARN-1896: -- Attachment: YARN-1896.v2.patch > For FairScheduler expose MinimumQueueResource of each queue in QueueMetrics > -

[jira] [Commented] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

[ https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989672#comment-13989672 ] Hadoop QA commented on YARN-1805: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

[ https://issues.apache.org/jira/browse/YARN-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989643#comment-13989643 ] Hadoop QA commented on YARN-2020: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

[ https://issues.apache.org/jira/browse/YARN-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yeqi updated YARN-2020: --- Attachment: YARN-2020.patch patch submitted > observeOnly should be checked before any preemption computation started

[jira] [Created] (YARN-2020) observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java

yeqi created YARN-2020: -- Summary: observeOnly should be checked before any preemption computation started inside containerBasedPreemptOrKill() of ProportionalCapacityPreemptionPolicy.java Key: YARN-2020 URL: https://issues.

[jira] [Updated] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

[ https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1805: -- Attachment: YARN-1805.patch The patch includes YARN-1803 and YARN-1897 for jenkins to build. > Signal container

[jira] [Assigned] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

[ https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma reassigned YARN-1805: - Assignee: Ming Ma > Signal container request delivery from resourcemanager to nodemanager > --

[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved

[ https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989571#comment-13989571 ] Hudson commented on YARN-1201: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1775 (See [htt

[jira] [Commented] (YARN-2019) Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989487#comment-13989487 ] Junping Du commented on YARN-2019: -- The bad news could be: the exception could be repeated

[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved

[ https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989451#comment-13989451 ] Hudson commented on YARN-1201: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1749 (See [https://

[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved

[ https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989424#comment-13989424 ] Hudson commented on YARN-1201: -- FAILURE: Integrated in Hadoop-Yarn-trunk #558 (See [https://b