[jira] [Created] (YARN-2324) Race condition in continuousScheduling for FairScheduler

zhihai xu created YARN-2324: --- Summary: Race condition in continuousScheduling for FairScheduler Key: YARN-2324 URL: https://issues.apache.org/jira/browse/YARN-2324 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-2324) Race condition in continuousScheduling for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067821#comment-14067821 ] Wei Yan commented on YARN-2324: --- duplicate of YARN-2273? > Race condition in continuousSched

[jira] [Assigned] (YARN-2324) Race condition in continuousScheduling for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu reassigned YARN-2324: --- Assignee: zhihai xu > Race condition in continuousScheduling for FairScheduler > -

[jira] [Updated] (YARN-2324) Race condition in continuousScheduling for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated YARN-2324: Attachment: YARN-2324.000.patch > Race condition in continuousScheduling for FairScheduler > ---

[jira] [Commented] (YARN-2324) Race condition in continuousScheduling for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067844#comment-14067844 ] Hadoop QA commented on YARN-2324: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes

[ https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-2013: - Attachment: YARN-2013.5.patch [~djp], thank you for review! Updated a patch to address your commen

[jira] [Commented] (YARN-2324) Race condition in continuousScheduling for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067889#comment-14067889 ] Tsuyoshi OZAWA commented on YARN-2324: -- [~zxu], thank you for reporting and your contr

[jira] [Commented] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes

[ https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067898#comment-14067898 ] Hadoop QA commented on YARN-2013: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes

[ https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067922#comment-14067922 ] Tsuyoshi OZAWA commented on YARN-2013: -- The test failure is not related. > The diagno

[jira] [Commented] (YARN-2309) NPE during RM-Restart test scenario

[ https://issues.apache.org/jira/browse/YARN-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067928#comment-14067928 ] Tsuyoshi OZAWA commented on YARN-2309: -- [~nishan], thank you for reporting. The patch

[jira] [Commented] (YARN-2313) Livelock can occur on FairScheduler when there are lots entry in queue

[ https://issues.apache.org/jira/browse/YARN-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067998#comment-14067998 ] Sandy Ryza commented on YARN-2313: -- Thanks for reporting this [~ozawa]. A couple nits: *

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068017#comment-14068017 ] Sandy Ryza commented on YARN-796: - I'm worried that the proposal is becoming too complex. C

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068024#comment-14068024 ] Craig Welch commented on YARN-2008: --- Hmm, I don't think that we can change headroom to be

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068026#comment-14068026 ] Chen He commented on YARN-2008: --- Thank you for the patch, [~cwelch] {quote} and if the applic

[jira] [Created] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler

zhihai xu created YARN-2325: --- Summary: need check whether node is null in nodeUpdate for FairScheduler Key: YARN-2325 URL: https://issues.apache.org/jira/browse/YARN-2325 Project: Hadoop YARN Iss

[jira] [Updated] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated YARN-2325: Attachment: YARN-2325.000.patch > need check whether node is null in nodeUpdate for FairScheduler > ---

[jira] [Assigned] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu reassigned YARN-2325: --- Assignee: zhihai xu > need check whether node is null in nodeUpdate for FairScheduler > -

[jira] [Commented] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068035#comment-14068035 ] Hadoop QA commented on YARN-2325: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068041#comment-14068041 ] Jian Fang commented on YARN-796: As Sandy pointed out, seems the scope becomes bigger and bi

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068043#comment-14068043 ] Allen Wittenauer commented on YARN-796: --- I agree pretty much completely with everythin

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068048#comment-14068048 ] Alejandro Abdelnur commented on YARN-796: - i agree with sandy and allen. said that

[jira] [Updated] (YARN-2211) RMStateStore needs to save AMRMToken master key for recovery when RM restart/failover happens

[ https://issues.apache.org/jira/browse/YARN-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-2211: Attachment: YARN-2211.5.1.patch Same patch with -1 on release audit fix > RMStateStore needs to save AMRMTo

[jira] [Commented] (YARN-2211) RMStateStore needs to save AMRMToken master key for recovery when RM restart/failover happens

[ https://issues.apache.org/jira/browse/YARN-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068069#comment-14068069 ] Hadoop QA commented on YARN-2211: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068074#comment-14068074 ] Jian He commented on YARN-1372: --- {code} NM informs RM and holds on to the information (YARN-1

[jira] [Updated] (YARN-2249) RM may receive container release request on AM resync before container is actually recovered

[ https://issues.apache.org/jira/browse/YARN-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-2249: -- Description: AM resync on RM restart will send outstanding container release requests back to the new RM. In the

[jira] [Commented] (YARN-2249) RM may receive container release request on AM resync before container is actually recovered

[ https://issues.apache.org/jira/browse/YARN-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068076#comment-14068076 ] Jian He commented on YARN-2249: --- One possible solution is to have AM always send the whole pe

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068116#comment-14068116 ] Wangda Tan commented on YARN-796: - Really thanks all your comments above, As Sandy, Alejand

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068120#comment-14068120 ] Alejandro Abdelnur commented on YARN-796: - Wangda, your usecase is throwing overboar

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068124#comment-14068124 ] Wangda Tan commented on YARN-796: - Hi Alejandro, I totally understand the use case I mentio

[jira] [Commented] (YARN-2323) FairShareComparator creates too much Resource object

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068143#comment-14068143 ] Sandy Ryza commented on YARN-2323: -- As it's a static final variable, ONE should be all cap

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068144#comment-14068144 ] Alejandro Abdelnur commented on YARN-796: - Wangda, i'm afraid i'm lost with your las

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068145#comment-14068145 ] Wangda Tan commented on YARN-796: - Alejandro, I think we've mentioned this in our design doc

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068147#comment-14068147 ] Wangda Tan commented on YARN-1198: -- I've just taken a look at all sub tasks of this JIRA,

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068148#comment-14068148 ] Wangda Tan commented on YARN-2008: -- Hi [~cwelch], [~airbots], I've put my comment on YARN-

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068163#comment-14068163 ] Wangda Tan commented on YARN-796: - Hi [~sunilg], bq. 2. Regarding reservations, how about in

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068169#comment-14068169 ] Allen Wittenauer commented on YARN-796: --- bq. An important use case we can see is, some

[jira] [Updated] (YARN-2323) FairShareComparator creates too much Resource object

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Zhiguo updated YARN-2323: -- Attachment: YARN-2323-2.patch patch revised according to [~sandyr]'s comments. > FairShareComparator cr

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068184#comment-14068184 ] Wangda Tan commented on YARN-796: - bq. You can solve this problem today by just running sepa

[jira] [Commented] (YARN-2323) FairShareComparator creates too much Resource object

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068187#comment-14068187 ] Sandy Ryza commented on YARN-2323: -- +1 > FairShareComparator creates too much Resource ob

[jira] [Commented] (YARN-2323) FairShareComparator creates too much Resource object

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068207#comment-14068207 ] Hadoop QA commented on YARN-2323: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (YARN-2323) FairShareComparator creates too many Resource objects

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2323: - Summary: FairShareComparator creates too many Resource objects (was: FairShareComparator creates too much

[jira] [Commented] (YARN-2323) FairShareComparator creates too many Resource objects

[ https://issues.apache.org/jira/browse/YARN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068210#comment-14068210 ] Hudson commented on YARN-2323: -- FAILURE: Integrated in Hadoop-trunk-Commit #5921 (See [https:

[jira] [Updated] (YARN-2033) Investigate merging generic-history into the Timeline Store

[ https://issues.apache.org/jira/browse/YARN-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2033: -- Attachment: YARN-2033_ALL.2.patch > Investigate merging generic-history into the Timeline Store > --

[jira] [Updated] (YARN-2033) Investigate merging generic-history into the Timeline Store

[ https://issues.apache.org/jira/browse/YARN-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2033: -- Attachment: YARN-2033.2.patch TestFSDownload is not related, but upload a patch to fix remaining issues.

[jira] [Updated] (YARN-2284) Find missing config options in YarnConfiguration and yarn-default.xml

[ https://issues.apache.org/jira/browse/YARN-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated YARN-2284: - Attachment: YARN2284-02.patch Second attempt. - Have minimal changes to Configuration to read in xml file

[jira] [Commented] (YARN-2033) Investigate merging generic-history into the Timeline Store

[ https://issues.apache.org/jira/browse/YARN-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068217#comment-14068217 ] Hadoop QA commented on YARN-2033: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-2284) Find missing config options in YarnConfiguration and yarn-default.xml

[ https://issues.apache.org/jira/browse/YARN-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068218#comment-14068218 ] Hadoop QA commented on YARN-2284: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068230#comment-14068230 ] Allen Wittenauer commented on YARN-796: --- Then let me be more blunt about it: I'm -1 t

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068248#comment-14068248 ] Wangda Tan commented on YARN-796: - Allen, I think what we was just talking about is how to s

[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens

[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2247: Attachment: apache-yarn-2247.3.patch {quote} bq.The current implementation uses the standard htt

[jira] [Commented] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler

[ https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068258#comment-14068258 ] Tsuyoshi OZAWA commented on YARN-2325: -- Hi [~zxu], could you explain the case when nod