[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary
[
https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726540#comment-16726540
]
Hadoop QA commented on YARN-8925:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
59s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
40s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
42s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
5s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} branch-3.2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 346 unchanged - 0 fixed = 348 total (was 346) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 34s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
17s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
22s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
35s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 67m
6s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 55s{color} |
{color:black}
[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[
https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726542#comment-16726542
]
Hadoop QA commented on YARN-5168:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
24m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 15m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
43s{color} | {color:green} root: The patch generated 0 new + 871 unchanged - 7
fixed = 871 total (was 878) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
22s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
7s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
43s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m
[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726545#comment-16726545 ] Weiwei Yang commented on YARN-8925: --- Looks good, committing now. > Updating distributed node attributes only when necessary > > > Key: YARN-8925 > URL: https://issues.apache.org/jira/browse/YARN-8925 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Labels: performance > Attachments: YARN-8925-branch-3.2.001.patch, YARN-8925.001.patch, > YARN-8925.002.patch, YARN-8925.003.patch, YARN-8925.004.patch, > YARN-8925.005.patch, YARN-8925.006.patch, YARN-8925.007.patch, > YARN-8925.008.patch, YARN-8925.009.patch, YARN-8925.010.patch > > > Currently if distributed node attributes exist, even though there is no > change, updating for distributed node attributes will happen in every > heartbeat between NM and RM. Updating process will hold > NodeAttributesManagerImpl#writeLock and may have some influence in a large > cluster. We have found nodes UI of a large cluster is opened slowly and most > time it's waiting for the lock in NodeAttributesManagerImpl. I think this > updating should be called only when necessary to enhance the performance of > related process. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9132) Add file permission check for auxiliary services manifest file
[ https://issues.apache.org/jira/browse/YARN-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727024#comment-16727024 ] Hudson commented on YARN-9132: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15652 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15652/]) YARN-9132. Added file permission check for auxiliary services manifest (eyang: rev ea724181d66ebe3c2cc7ed071948e9bc463bf223) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestAuxServices.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/AuxServices.java > Add file permission check for auxiliary services manifest file > -- > > Key: YARN-9132 > URL: https://issues.apache.org/jira/browse/YARN-9132 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9132.1.patch, YARN-9132.2.patch, YARN-9132.3.patch > > > The manifest file in HDFS must be owned by YARN admin or YARN service user > only. This check helps to prevent loading of malware into node manager JVM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727164#comment-16727164 ] Hudson commented on YARN-5168: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15654 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15654/]) YARN-5168. Added exposed port information for Docker container. (eyang: rev f82922dcfae0b7e3e68ce970c441d8b7edc59acf) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-app-attempt.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-timeline-container.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppAttemptInfo.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-container.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-component-instance.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerReport.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-component-instance.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/timelineservice/ServiceTimelinePublisher.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerUpdateContext.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV2Publisher.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainer.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ContainerMetricsConstants.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/timelineservice/ServiceTimelineMetricsConstants.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerStatusPBImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppAttempts.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/UpdatedContainerInfo.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-container.js * (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDSAppMaster.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/MockContainer.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/records/ContainerHistoryData.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/runtime/ContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerStatus.java * (edit)
[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726548#comment-16726548 ] Weiwei Yang commented on YARN-8925: --- I've committed this to trunk and branch-3.2. Thanks for the contribution [~Tao Yang]. > Updating distributed node attributes only when necessary > > > Key: YARN-8925 > URL: https://issues.apache.org/jira/browse/YARN-8925 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Labels: performance > Fix For: 3.3.0, 3.2.1 > > Attachments: YARN-8925-branch-3.2.001.patch, YARN-8925.001.patch, > YARN-8925.002.patch, YARN-8925.003.patch, YARN-8925.004.patch, > YARN-8925.005.patch, YARN-8925.006.patch, YARN-8925.007.patch, > YARN-8925.008.patch, YARN-8925.009.patch, YARN-8925.010.patch > > > Currently if distributed node attributes exist, even though there is no > change, updating for distributed node attributes will happen in every > heartbeat between NM and RM. Updating process will hold > NodeAttributesManagerImpl#writeLock and may have some influence in a large > cluster. We have found nodes UI of a large cluster is opened slowly and most > time it's waiting for the lock in NodeAttributesManagerImpl. I think this > updating should be called only when necessary to enhance the performance of > related process. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726565#comment-16726565 ] Xun Liu commented on YARN-5168: --- [~eyang], Please help me review the code, thank you! :) > Add port mapping handling when docker container use bridge network > -- > > Key: YARN-5168 > URL: https://issues.apache.org/jira/browse/YARN-5168 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jun Gong >Assignee: Xun Liu >Priority: Major > Labels: Docker > Attachments: YARN-5168.001.patch, YARN-5168.002.patch, > YARN-5168.003.patch, YARN-5168.004.patch, YARN-5168.005.patch, > YARN-5168.006.patch, YARN-5168.007.patch, YARN-5168.008.patch, > YARN-5168.009.patch, YARN-5168.010.patch, YARN-5168.011.patch, > YARN-5168.012.patch, YARN-5168.013.patch, YARN-5168.014.patch, > YARN-5168.015.patch, YARN-5168.016.patch, YARN-5168.017.patch, > YARN-5168.018.patch, YARN-5168.019.patch, YARN-5168.020.patch, > exposedPorts1.png, exposedPorts2.png > > > YARN-4007 addresses different network setups when launching the docker > container. We need support port mapping when docker container uses bridge > network. > The following problems are what we faced: > 1. Add "-P" to map docker container's exposed ports to automatically. > 2. Add "-p" to let user specify specific ports to map. > 3. Add service registry support for bridge network case, then app could find > each other. It could be done out of YARN, however it might be more convenient > to support it natively in YARN. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9108) FederationIntercepter merge home and second response local variable spell mistake
[
https://issues.apache.org/jira/browse/YARN-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Modi updated YARN-9108:
Attachment: YARN-9108.003.patch
> FederationIntercepter merge home and second response local variable spell
> mistake
> -
>
> Key: YARN-9108
> URL: https://issues.apache.org/jira/browse/YARN-9108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.3.0
>Reporter: Morty Zhong
>Assignee: Abhishek Modi
>Priority: Minor
> Attachments: YARN-9108.001.patch, YARN-9108.002.patch,
> YARN-9108.003.patch
>
>
> method 'mergeAllocateResponse' in class FederationIntercepter.java line 1315
> the left variable `par2` should be `par1`
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par2.getContainers().addAll(par2.getContainers());
> }
> {code}
> should be
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par1.getContainers().addAll(par2.getContainers());//edited line
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9038) [CSI] Add ability to publish/unpublish volumes on node managers
[ https://issues.apache.org/jira/browse/YARN-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated YARN-9038: -- Attachment: YARN-9038.007.patch > [CSI] Add ability to publish/unpublish volumes on node managers > --- > > Key: YARN-9038 > URL: https://issues.apache.org/jira/browse/YARN-9038 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Labels: CSI > Attachments: YARN-9038.001.patch, YARN-9038.002.patch, > YARN-9038.003.patch, YARN-9038.004.patch, YARN-9038.005.patch, > YARN-9038.006.patch, YARN-9038.007.patch > > > We need to add ability to publish volumes on node managers in staging area, > under NM's local dir. And then mount the path to docker container to make it > visible in the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9038) [CSI] Add ability to publish/unpublish volumes on node managers
[
https://issues.apache.org/jira/browse/YARN-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726662#comment-16726662
]
Weiwei Yang commented on YARN-9038:
---
Hi [~sunilg]
Thanks for the review.
{quote}{{csiDriverName}} value could be const string than a simple plain string.
{quote}
{{csiDriverName}} should be a configurabled value, I have a TODO there. I plan
to get it done in YARN-9086, the AUX service for csi-driver-adaptors will be a
set of services, in order to support multiple drivers. And their driver names
will be loading from configuration. Here is to let code work with
ch.ctrox.csi.s3-driver.
{quote} some logs could be debug such as translated message ones.
{quote}
Sure. Changed to DEBUG level.
{quote}could CsiGrpcClient be cached?
{quote}
The client once created, it will create a netty channel and connect to a socket
address, so if we cache it that will waste some resource. Since volume related
call won't be very frequent, I think we are fine to create it on the fly.
{quote}cap and cap1 names could be more clearer.
{quote}
Yes, I renamed \{{cap1}} to \{{csiVolumeCap}}.
{quote}Is that persistence needed?
{quote}
I think so. This is on the node manager side, on RM side we are stateless.
Because CSI protocol ensures each calls are *idempotent*. That means, e.g we
can call publish volume multiple times for a given volume and it will return us
success all the time. This design makes us easier to do recovery. However on
the NM side, as we need to memorize the location of the local volume root, we
need to store that data in the container. Then we need to do persistent so we
are able to retrieve it on recovery.
I have uploaded a new patch, pls take a look.
Thanks.
> [CSI] Add ability to publish/unpublish volumes on node managers
> ---
>
> Key: YARN-9038
> URL: https://issues.apache.org/jira/browse/YARN-9038
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Labels: CSI
> Attachments: YARN-9038.001.patch, YARN-9038.002.patch,
> YARN-9038.003.patch, YARN-9038.004.patch, YARN-9038.005.patch,
> YARN-9038.006.patch
>
>
> We need to add ability to publish volumes on node managers in staging area,
> under NM's local dir. And then mount the path to docker container to make it
> visible in the container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9038) [CSI] Add ability to publish/unpublish volumes on node managers
[
https://issues.apache.org/jira/browse/YARN-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726610#comment-16726610
]
Sunil Govindan commented on YARN-9038:
--
Thanks [~cheersyang]. Some more comments
# In CsiAdaptorProtocolService ctor, {{csiDriverName}} value could be const
string than a simple plain string.
# In CsiAdaptorProtocolService#nodePublishVolume, some logs could be debug
such as translated message ones.
# In CsiClientImpl, could CsiGrpcClient be cached?
# In NodePublishVolumeRequestProtoTranslator, cap and cap1 names could be more
clearer.
# recordContainerCsiVolumesRootDir, i saw a todo. Is that persistence needed?
Will RM have a cache of allocated and mapped volumes?
> [CSI] Add ability to publish/unpublish volumes on node managers
> ---
>
> Key: YARN-9038
> URL: https://issues.apache.org/jira/browse/YARN-9038
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Labels: CSI
> Attachments: YARN-9038.001.patch, YARN-9038.002.patch,
> YARN-9038.003.patch, YARN-9038.004.patch, YARN-9038.005.patch,
> YARN-9038.006.patch
>
>
> We need to add ability to publish volumes on node managers in staging area,
> under NM's local dir. And then mount the path to docker container to make it
> visible in the container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726745#comment-16726745
]
Akira Ajisaka commented on YARN-9154:
-
+1
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch, screenshot-1.png
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-9154:
---
Attachment: screenshot-1.png
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch, screenshot-1.png
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726701#comment-16726701
]
Ayush Saxena commented on YARN-9154:
Thanx [~ajisakaa] for putting this up Uploaded patch with the fix.
Pls Review!!!
Following is the screenshot after the fix:
!screenshot-1.png!
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch, screenshot-1.png
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9108) FederationIntercepter merge home and second response local variable spell mistake
[
https://issues.apache.org/jira/browse/YARN-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Modi updated YARN-9108:
Attachment: YARN-9108.004.patch
> FederationIntercepter merge home and second response local variable spell
> mistake
> -
>
> Key: YARN-9108
> URL: https://issues.apache.org/jira/browse/YARN-9108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.3.0
>Reporter: Morty Zhong
>Assignee: Abhishek Modi
>Priority: Minor
> Attachments: YARN-9108.001.patch, YARN-9108.002.patch,
> YARN-9108.003.patch, YARN-9108.004.patch
>
>
> method 'mergeAllocateResponse' in class FederationIntercepter.java line 1315
> the left variable `par2` should be `par1`
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par2.getContainers().addAll(par2.getContainers());
> }
> {code}
> should be
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par1.getContainers().addAll(par2.getContainers());//edited line
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9108) FederationIntercepter merge home and second response local variable spell mistake
[
https://issues.apache.org/jira/browse/YARN-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726787#comment-16726787
]
Hadoop QA commented on YARN-9108:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 47s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 36s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9108 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12952683/YARN-9108.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b39dd0a32e27 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f659485 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/22944/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22944/testReport/ |
| Max. process+thread count | 413 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
[jira] [Updated] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-9154:
---
Attachment: YARN-9154-01.patch
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reassigned YARN-9154:
--
Assignee: Ayush Saxena
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726709#comment-16726709
]
Hadoop QA commented on YARN-9154:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-9154 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9154 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22943/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch, screenshot-1.png
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9108) FederationIntercepter merge home and second response local variable spell mistake
[
https://issues.apache.org/jira/browse/YARN-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726729#comment-16726729
]
Hadoop QA commented on YARN-9108:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
42s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 25s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 18s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9108 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12952671/YARN-9108.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 643c31385898 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f659485 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/22942/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit |
[jira] [Commented] (YARN-9154) Fix itemization in YARN service quickstart document
[
https://issues.apache.org/jira/browse/YARN-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727212#comment-16727212
]
Hudson commented on YARN-9154:
--
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #15655 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/15655/])
YARN-9154. Fix itemization in YARN service quickstart document. (aajisaka: rev
e98e963e5143600017d2d227d504140b8802439d)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/QuickStart.md
> Fix itemization in YARN service quickstart document
> ---
>
> Key: YARN-9154
> URL: https://issues.apache.org/jira/browse/YARN-9154
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ayush Saxena
>Priority: Minor
> Labels: newbie
> Fix For: 3.1.2, 3.3.0, 3.2.1
>
> Attachments: Screen Shot 2018-12-20 at 15.54.16.png,
> YARN-9154-01.patch, screenshot-1.png
>
>
> {noformat:title=QuickStart.md}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> should be
> {noformat}
> Params:
> - SERVICE_NAME: The name of the service. Note that this needs to be unique
> across running services for the current user.
> - PATH_TO_SERVICE_DEF: The path to the service definition file in JSON format.
> {noformat}
> to render correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9108) FederationIntercepter merge home and second response local variable spell mistake
[
https://issues.apache.org/jira/browse/YARN-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726958#comment-16726958
]
Íñigo Goiri commented on YARN-9108:
---
{{TestFederationInterceptor}} looks very suspicious.
Any ideas if it is related?
> FederationIntercepter merge home and second response local variable spell
> mistake
> -
>
> Key: YARN-9108
> URL: https://issues.apache.org/jira/browse/YARN-9108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.3.0
>Reporter: Morty Zhong
>Assignee: Abhishek Modi
>Priority: Minor
> Attachments: YARN-9108.001.patch, YARN-9108.002.patch,
> YARN-9108.003.patch, YARN-9108.004.patch
>
>
> method 'mergeAllocateResponse' in class FederationIntercepter.java line 1315
> the left variable `par2` should be `par1`
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par2.getContainers().addAll(par2.getContainers());
> }
> {code}
> should be
> {code:java}
> if (par1 != null && par2 != null) {
> par1.getResourceRequest().addAll(par2.getResourceRequest());
> par1.getContainers().addAll(par2.getContainers());//edited line
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4714) [Java 8] Over usage of virtual memory
[ https://issues.apache.org/jira/browse/YARN-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726879#comment-16726879 ] Andres Namm commented on YARN-4714: --- I have built another answer which depends whether you are using spark client or cluster mode. * In cluster mode it failed when I specified *Driver Memory* --driver-memory to be 512m. (The default setting requested 2GB of am resources (This consists of driver memory + Overhead requested for Application Master) which was enough) * In client mode the setting that mattered was spark.yarn.am.memory as by default this requested only 1024m for the AM which is too little as Java 8 requires a lot of virtual memory. > 1024m seemed to be working. Answer is described [here|https://github.com/AndresNamm/SparkConfAndDebugging/blob/master/Debug/SparkMemoryIssue.md] > [Java 8] Over usage of virtual memory > - > > Key: YARN-4714 > URL: https://issues.apache.org/jira/browse/YARN-4714 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Mohammad Kamrul Islam >Assignee: Mohammad Kamrul Islam >Priority: Blocker > Attachments: HADOOP-11364.01.patch > > > In our Hadoop 2 + Java8 effort , we found few jobs are being Killed by Hadoop > due to excessive virtual memory allocation. Although the physical memory > usage is low. > The most common error message is "Container [pid=??,containerID=container_??] > is running beyond virtual memory limits. Current usage: 365.1 MB of 1 GB > physical memory used; 3.2 GB of 2.1 GB virtual memory used. Killing > container." > We see this problem for MR job as well as in spark driver/executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
