[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997807#comment-14997807
]
Naganarasimha G R commented on YARN-4338:
-
IMHO, i think its worth a try as anyway null is treated as default Label so
funcationally its fine. Even if it fails I expect some test cases failing but
it will prevent future testcases not require to handle this explicitly.
Thoughts ?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997808#comment-14997808
]
Naganarasimha G R commented on YARN-4338:
-
IMHO, i think its worth a try as anyway null is treated as default Label so
funcationally its fine. Even if it fails I expect some test cases failing but
it will prevent future testcases not require to handle this explicitly.
Thoughts ?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997806#comment-14997806
]
Naganarasimha G R commented on YARN-4338:
-
IMHO, i think its worth a try as anyway null is treated as default Label so
funcationally its fine. Even if it fails I expect some test cases failing but
it will prevent future testcases not require to handle this explicitly.
Thoughts ?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997654#comment-14997654
]
Wangda Tan commented on YARN-4338:
--
Thanks for comments: [~sunilg]/[~Naganarasimha].
[~xinwei], I would prefer to keep main logic as-is and fix tests, the major
concern is people may think node label expression required to check null in CS
logic, which could reduce code readability. I'm OK with common code (such as
AppSchedulingInfo) check null for nodeLabelExpression. Could you fix tests of
YARN-2618 instead of updating RegularContainerAllocator?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996185#comment-14996185
]
Sunil G commented on YARN-4338:
---
Recently in YARN-4250, there were a chance that
{{anyRequest.getNodeLabelExpression()}} become null becaue
ApplicationMasterService may not normalizes expression always.
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996191#comment-14996191
]
Naganarasimha G R commented on YARN-4338:
-
Hi [~xinwei],
What was the scenario in which you got this NPE?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996241#comment-14996241
]
Naganarasimha G R commented on YARN-4338:
-
Hi [~xinwei],
In that case its expected to come through ApplicationMasterService , so may be
its sufficient to rectify the test case with default label "" .
[~sunilg] & [~wangda]
But as we are coming across this more frequently, how about correcting it with
setting with Default Label when using other overloaded methods or even in the
main overloaded method we can check for null and set to Default i.e. "" ?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996338#comment-14996338
]
Naganarasimha G R commented on YARN-4338:
-
meant => ??how about correcting it in *ResourceRequest* with setting Default
Label when using other overloaded methods or in the main overloaded method we
can check for null and set to Default label i.e. ""??
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997663#comment-14997663
]
Wangda Tan commented on YARN-4338:
--
I don't know what's the impact of it, since we're leveraging
nodeLabelExpression=="null" to represent "unset" in ResourceRequest. I think
some code path will fail if ResourceRequest.getNodeLabelExpression returns ""
if it == null/
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997659#comment-14997659
]
Naganarasimha G R commented on YARN-4338:
-
Hi [~wangda],
How about setting Default Label in ResourceRequest when not set ?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996194#comment-14996194
]
Sunil G commented on YARN-4338:
---
Missed to add a point earlier, are you using a custom scheduler here?
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4338) NPE in RegularContainerAllocator.preCheckForNewContainer()
[
https://issues.apache.org/jira/browse/YARN-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996246#comment-14996246
]
Xinwei Qin commented on YARN-4338:
---
Thanks [~Naganarasimha] for your suggestion, the test case passed with this
modification.
> NPE in RegularContainerAllocator.preCheckForNewContainer()
> --
>
> Key: YARN-4338
> URL: https://issues.apache.org/jira/browse/YARN-4338
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xinwei Qin
>Priority: Minor
> Attachments: YARN-4338.001.patch
>
>
> The codes in RegularContainerAllocator.preCheckForNewContainer():
> {code}
> if (anyRequest.getNodeLabelExpression()
> .equals(RMNodeLabelsManager.NO_LABEL)) {
> missedNonPartitionedRequestSchedulingOpportunity =
> application
> .addMissedNonPartitionedRequestSchedulingOpportunity(priority);
> }
> {code}
> {code}anyRequest.getNodeLabelExpression(){code}may return null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
