[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894020#comment-16894020 ] Daniel Templeton commented on YARN-8480: I still hold that boolean resources are the way to go, but given that I'm no longer active in YARN, and that Yunicorn aspires to be the scheduler to rule them all, it's not worth the fight. Go ahead and close it. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[
https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885134#comment-16885134
]
Hadoop QA commented on YARN-8480:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} YARN-8480 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-8480 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12930979/YARN-8480.002.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/24395/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add boolean option for resources
>
>
> Key: YARN-8480
> URL: https://issues.apache.org/jira/browse/YARN-8480
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Daniel Templeton
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-8480.001.patch, YARN-8480.002.patch
>
>
> Make it possible to define a resource with a boolean value.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885128#comment-16885128 ] Szilard Nemeth commented on YARN-8480: -- Hi [~leftnoteasy], [~templedf], [~sunilg]! Given that we had a useful discussion here, but the rest of the opinions tend to point to node attributes as the preferred approach over boolean resources, I'm fine by closing this jira off. What do you think? Thanks! > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552196#comment-16552196 ] Naganarasimha G R commented on YARN-8480: - Sorry for pitching in late, ran through the discussions in the Jira and I concur with [~leftnoteasy], I do not see the need for providing 2 approaches to address the same scenario. Understand there would be some initial hickups in using Node Attributes in FS but i think that would be the better approach than having 2 different approaches for FS and CS to address the same scenario. If we wear the user's cap and see it would be good to have a single approach to address a scenario irrespective of the scheduler. If we start addressing scenario's differently for FS and CS then there will 2 parallel systems within YARN which is not desirable. [~templedf],[~snemeth], May be if you guys can share all the features which you want from Node Attributes to address then we can confirm to what extent we have already covered in Phase 1. As far as i see if its only exist and not exist then we do already cover. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[
https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551245#comment-16551245
]
Wangda Tan commented on YARN-8480:
--
[~templedf],
{quote}It also looks to me like the PCM is an out-of-band scheduling process
that skips the usual scheduling process.
{quote}
There're two approaches to do PCM allocation, one is called
{{PlacementConstraintProcessor}} it is out-of-band. And the other one is
scheduler, which happens inside normal scheduler loop. You can check
http://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html
for more details.
When we were doing scheduler changes to placement constraint, we put most
logics to common part like AppSchedulingInfo, so other scheduler impl can use
it directly. The hardest part is done, you only add some glue code to make FS
work.
If you really don't like to utilize the new SchedulingRequest, adding node
attribute to ResourceRequest could be also considered (I'm not quite like the
approach, but it is acceptable to me).
By going with the approach to support node attribute by ResourceRequest,
changes in FS should be minimum and all other logics like how to assign node
attribute to nodes, how node to report node attributes to RM, how web UIs
renders attributes etc. can be reused.
Looping other folks who are doing node attribute support here for more
thoughts: [~sunilg], [~cheersyang], [~Naganarasimha], [~bibinchundatt].
> Add boolean option for resources
>
>
> Key: YARN-8480
> URL: https://issues.apache.org/jira/browse/YARN-8480
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Daniel Templeton
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-8480.001.patch, YARN-8480.002.patch
>
>
> Make it possible to define a resource with a boolean value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551228#comment-16551228 ] Daniel Templeton commented on YARN-8480: I read through both node attributes and rich placement constraints. Correct me if I'm wrong, but it looks to me like node attributes rely on the PCM from rich placement constraints for scheduling. It also looks to me like the PCM is an out-of-band scheduling process that skips the usual scheduling process. The amount of code that changed in YARN-7670 to support rich placement constraints in capacity scheduler was smallish, but the impact is fairly large. Yes, the patch for this JIRA does touch a lot of common files, but the changes are minor and non-invasive and will come with careful testing for performance impact. The patch already includes a huge amount of new unit tests (on top of what was added with resource types already) to make sure nothing is impacted as a result of this minor extension. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541203#comment-16541203 ] Weiwei Yang commented on YARN-8480: --- Thanks [~leftnoteasy] for the info. I agree this is overlapped with node-attributes, and a sub-set of what node-attributes can support. We can be more flexible with node-attributes, e.g to specify java_version > 1.7 (we use this for library dependency constraints). As YARN-3409 is close to be done, is it an easier way to enhance FS to support placement constraints? It's not a big change in scheduler code. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541156#comment-16541156 ] Wangda Tan commented on YARN-8480: -- [~cheersyang], What [~templedf] / [~snemeth] proposed is make the resource like a label, for example, node can report it has resource: memory=2048,vcore=3,has_java=true. And AM can request resource with ...has_java=true. Allocating container on the node will not update node's available resource, as far as the node has enough memory and vcores, it can allocate >1 containers with has_java=true resource request. This is an alternative way to represent node label by using resource types. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541039#comment-16541039 ] Weiwei Yang commented on YARN-8480: --- Hi [~templedf]/[~snemeth] Could you please elaborate the use case where you use this boolean type resource? What is the difference with a countable resource with value only 0 or 1? Thanks > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540897#comment-16540897 ] Wangda Tan commented on YARN-8480: -- btw, [~templedf], I knew you found some troubles of support node partition concept in FS (YARN-2497). The node attribute should be a much easier support because there's no resource sharing required under node attribute, everything comes with FCFS. Basically you should only add a if check before deciding allocate container X on node Y. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540895#comment-16540895 ] Wangda Tan commented on YARN-8480: -- [~templedf], If this only changes Fair Scheduler, I'm fine with that. However this touches Resource/ResourceInformation/RMAminCLI/ResourceCalculator/CapacityScheduler-implementation, etc. If you could take a closer look at YARN-3409 APIs, it should not be hard at all. Should definitely cheaper than adding a new resource type. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540885#comment-16540885 ] Daniel Templeton commented on YARN-8480: I don't disagree that in terms of general use cases there's overlap between boolean resources and node attributes. From the perspective of fair scheduler, however, there's a big difference. Boolean resources are a simple mechanism for a pure label resource that is a minor extension of existing capabilities. Node labels or node attributes will be a major integration effort. While much of this work touches the common code, what we're proposing here is not intended as a general solution. Capacity scheduler is where the work for node attributes is happening, and we don't propose to change that or muddy those waters. This JIRA is intended only to provide a pure label resource for fair scheduler that isn't as heavy. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[
https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540810#comment-16540810
]
Wangda Tan commented on YARN-8480:
--
[~snemeth]/[~templedf].
To me we should move this to node attribute: we already have most things ready
here under YARN-3409 branch. I don't think we should duplicate the same things
in this JIRA.
For stuffs added to resource class, it's not necessarily to be countable,
(definition of countable according to design doc of YARN-3926):
{code}
When we speak of countable resources, we refer to resourcetypes where the
allocation and release of resources is a simple subtraction and addition
operation.
{code}
I'm supportive to add new resource types like set, range, or hierarchical (like
disk resource isolation which is mentioned by [~cheersyang]). The pure label
resource type should go to node attribute / node partition.
> Add boolean option for resources
>
>
> Key: YARN-8480
> URL: https://issues.apache.org/jira/browse/YARN-8480
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Daniel Templeton
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-8480.001.patch, YARN-8480.002.patch
>
>
> Make it possible to define a resource with a boolean value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[
https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538579#comment-16538579
]
genericqa commented on YARN-8480:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 20 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 24s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
26s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
47s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
19s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 69m
4s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m
5s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}190m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| |
org.apache.hadoop.yarn.conf.YarnConfiguration.DEFAULT_RM_CONFIGURATION_PROVIDER_CLASS
isn't final but should be
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538344#comment-16538344 ] Szilard Nemeth commented on YARN-8480: -- Uploaded second patch that fixes the trailing whitespace issues and one findbugs issue. The other findbugs issue that complains about DEFAULT_RM_CONFIGURATION_PROVIDER_CLASS should be final could not be fixed as the static initializer method in TestResource should modify this field in order to work correctly. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch, YARN-8480.002.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[
https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536795#comment-16536795
]
genericqa commented on YARN-8480:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 20 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 41s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 62 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
24s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
11s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 4s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m
9s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 13s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| |
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536640#comment-16536640 ] Szilard Nemeth commented on YARN-8480: -- Hi, I understand that we need to come to a conclusion about boolean resource types, but in the meantime you can check my patch as well that implements this feature. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-8480.001.patch > > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535536#comment-16535536 ] Zian Chen commented on YARN-8480: - [~templedf] , would you mind give some example what kind of resources can be treated as "boolean" resource here? To me, node labels and node attributes are orthogonal with each other. Node labels are addressing requirements that we want to specify apps running on the specific node with the resource they need, or partition cluster on organizations or workloads. While node attributes are solving the problem when we need to characterize a node based on many of its attributes and select a node based on expression of these attributes during scheduling. If we could find the user scenarios for boolean resource which can not be covered by node labels or node attributes very well. That will be a very good start for supporting this feature. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535146#comment-16535146 ] Sunil Govindan commented on YARN-8480: -- I think boolean resource is a bit vague item as per me. Currently we are considering only COUNTABLE resources. And you have mentioned that boolean resource is not consumable, hence its more or less falling in to NodeAttribute itself. Node attributes doesnt hold any resources to it unlike resources/labels. I think non-countable and sharable resources in YARN is pretty much an item to be discussed for its usage etc. Since it affects the resource calculator, we need to be clear on how it will used in normal calculations. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535129#comment-16535129 ] Daniel Templeton commented on YARN-8480: A boolean resource wouldn't be consumed by its use. The node attributes work is definitely in the same vein, though much heavier weight. The other difference is that there's no dependency on node labels or placement constraints here, so the scheduler implementation cost is very low. There is some overlap with node attributes, but when we have three entirely distinct ways to specify resources (resources, node labels, and node attributes), that's not too surprising. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8480) Add boolean option for resources
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527962#comment-16527962 ] Jason Lowe commented on YARN-8480: -- Could someone elaborate a bit more on the use-case for this and how having a nodemanager only report having a single instance of the resource using the existing, countable resources would not be sufficient? Is it that the boolean value is not "exhausted" when allocated? If so then this sounds like it overlaps quite a bit with the node attributes work in YARN-3409. > Add boolean option for resources > > > Key: YARN-8480 > URL: https://issues.apache.org/jira/browse/YARN-8480 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Daniel Templeton >Assignee: Szilard Nemeth >Priority: Major > > Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
