[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241057#comment-16241057 ] Hudson commented on YARN-5993: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13193 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13193/]) YARN-5993. Allow native services quicklinks to be exported for each (jianhe: rev a5e20f0fc1262dc57396f95c9d741b4486c728c2) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/common/tools/SliderUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/core/registry/docstore/ExportEntry.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/providers/docker/DockerProviderService.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/core/registry/docstore/PublishedExports.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/server/appmaster/web/view/IndexBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/providers/ProviderUtils.java > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch, > YARN-5993-yarn-native-services.004.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771253#comment-15771253 ] Gour Saha commented on YARN-5993: - +1 for the 004 patch. > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch, > YARN-5993-yarn-native-services.004.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771207#comment-15771207 ] Gour Saha commented on YARN-5993: - Then I like your strategy even more now. bq. It looks like the services API only supports global exports at this point (application-level quicklinks) and ignores any component quicklinks. On this point, current Slider design is that global exports are an aggregation of all component level quicklinks. Hence services API simply exposes the application level quicklinks. > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch, > YARN-5993-yarn-native-services.004.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771165#comment-15771165 ] Gour Saha commented on YARN-5993: - [~billie.rinaldi] I think we should file a sub-task under both, since work is required on both the ends (and make necessary links to each other). > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch, > YARN-5993-yarn-native-services.004.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771002#comment-15771002
]
Hadoop QA commented on YARN-5993:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
20s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 4 new + 171 unchanged - 6 fixed = 175 total (was 177)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | YARN-5993 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12844453/YARN-5993-yarn-native-services.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 77a1e3c6161c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / e9716f5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/14442/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/14442/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
|
| Console output |
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770818#comment-15770818 ] Billie Rinaldi commented on YARN-5993: -- I created YARN-6023. Wasn't sure if this should be a subtask of YARN-5079 or YARN-4757. I'll take a look at the checkstyle issue. > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770785#comment-15770785 ] Billie Rinaldi commented on YARN-5993: -- Yes, that's correct. These can be deduped on the exported value (this is already being done in the published configurations that the services API is using to retrieve the quicklinks). > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770579#comment-15770579 ] Gour Saha commented on YARN-5993: - One more point - bq. What is happening here is that exports can be specified at the global or the component level, but they are always exported by individual components. I like this idea. However what happens when an export at the global level does not contain any component specific placeholders? Will all components export it? > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[ https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770556#comment-15770556 ] Gour Saha commented on YARN-5993: - [~billie.rinaldi] the 003 patch looks good. There are 5 new checkstyle issues for 003. I think we can ignore the ones from equals method since the default eclipse one also generates similar code. However I think we should look at the one in PublishedExports.java line 105. Also, for the following - bq. If we want to support multiple IPs in the ServiceRecord, we'll need to open a separate ticket for that. Let's file a bug on this, because it is a valid usecase for a container to have a single hostname but multiple IPs. We can look into doing the right thing when we merging the DNS branch. > Allow native services quicklinks to be exported for each component > -- > > Key: YARN-5993 > URL: https://issues.apache.org/jira/browse/YARN-5993 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-5993-yarn-native-services.001.patch, > YARN-5993-yarn-native-services.002.patch, > YARN-5993-yarn-native-services.003.patch > > > The quicklinks export capability changed in switching from the agent provider > to the docker provider, and currently the docker provider only allows one > component to export links. We should improve this capability to be more in > line with the previous agent capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769123#comment-15769123
]
Hadoop QA commented on YARN-5993:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
58s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 5 new + 170 unchanged - 6 fixed = 175 total (was 176)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
38s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | YARN-5993 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843950/YARN-5993-yarn-native-services.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 71973911edaa 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 27a13ae |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/14438/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/14438/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
|
| Console output |
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762325#comment-15762325
]
Hadoop QA commented on YARN-5993:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
12s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
in yarn-native-services has 6 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
26s{color} | {color:red} hadoop-yarn-slider-core in yarn-native-services
failed. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 5 new + 170 unchanged - 6 fixed = 175 total (was 176)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-slider-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | YARN-5993 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843950/YARN-5993-yarn-native-services.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b57915817408 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / af649d2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/14367/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core-warnings.html
|
| javadoc |
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762261#comment-15762261
]
Hadoop QA commented on YARN-5993:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-5993 does not apply to yarn-native-services. Rebase
required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-5993 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843949/YARN-5993-yarn-native-services.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/14366/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Allow native services quicklinks to be exported for each component
> --
>
> Key: YARN-5993
> URL: https://issues.apache.org/jira/browse/YARN-5993
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
> Attachments: YARN-5993-yarn-native-services.001.patch,
> YARN-5993-yarn-native-services.002.patch
>
>
> The quicklinks export capability changed in switching from the agent provider
> to the docker provider, and currently the docker provider only allows one
> component to export links. We should improve this capability to be more in
> line with the previous agent capability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15757531#comment-15757531
]
Billie Rinaldi commented on YARN-5993:
--
bq.
org.apache.slider.providers.docker.DockerProviderService.publishExportGroups(String,
String, String, String, List) makes inefficient use of keySet iterator instead
of entrySet iterator
My patch does not introduce this issue and it is being fixed in YARN-5967.
bq. Why is this method synchronized now?
I think I was just being careful and don't recall a stronger justification. I
will remove it.
bq. Since we are discarding all but the first IP, I am thinking that we should
probably introduce a IPS_KEY_FORMAT of ${%s_IPS} and replace it with a comma
separated list of IPs when there are more than one IP for a single container.
This gives application owners a placeholder to fetch all the IPs as well (if
they want to).
I experimented with HOSTS and IPS formats in addition to HOST and IP.
Ultimately I found it to be more confusing to configure than just having HOST
and IP substitutions and having a separate export for each HOST and IP.
Currently in this patch each container can only export one export entry for a
given export name. We could allow a container to export multiple entries, one
for each IP, in the case where multiple IPs are returned in the container
status. However, it should be noted that we would still have only one
ServiceRecord registered in ZooKeeper. I am not aware of Registry DNS
supporting multiple IPs per ServiceRecord. If we want to support multiple IPs
in the ServiceRecord, we'll need to open a separate ticket for that.
bq. Just to confirm, if roleNameKey is null then there is no roleGroupKey,
right?
Let's say we have an external app external-app-name and that external app has a
unique component COMP with instances COMP1 and COMP2. When used in an assembly
app, the role names will be external-app-name-COMP1 and
external-app-name-COMP2. The role group for both will be
external-app-name-COMP. For the exports for these components, we are trying to
extract the roleNameKeys COMP1 and COMP2, and the roleNameGroup COMP, because
the external app's exports will have variables like $\{COMP1_HOST} and
$\{COMP_HOST}, without the external-app-name- prefix. In reality, roleNameKey
will always be roleGroupKey plus an integer. Neither will ever be null because
the role name and role group will always begin with the role prefix. The null
checks are just sanity checks.
bq. I think we should not block here, since it will help application owners to
know which variables were not substituted, rather than thinking that exports
don’t work at all. What do you think?
What is happening here is that exports can be specified at the global or the
component level, but they are always exported by individual components. The
tradeoff we are making is that users can put any exports they want at the
global level, and the provider will determine which component is supposed to
export them. I thought this would be easier for the user. (In the old agent
model, the user had to explicitly specify which components published which
exports.)
For example, if http://$\{COMP1_HOST}:port were specified at the global level,
only COMP1 will publish this export in the current patch.
If I make the change you suggest, COMP2 and COMP3 would also export
http://$\{COMP1_HOST}:port without the HOST variable filled in.
If we'd like to move to a model of having components publish all exports they
can see, I think we would have to disallow exports at the global level,
requiring all exports to be component properties. It looks like the services
API only supports global exports at this point (application-level quicklinks)
and ignores any component quicklinks.
bq. Please add null check for containerId in both the methods
bq. Change sortedEntries type from TreeMap to Map
bq. Remove Iterator import
bq. We can change values type from TreeSet to Set
bq. Can you rename entrySet to entries or exportEntries
bq. Can we rename this method to getExportEntries(String key)
bq. Check entry.getValue() for null
Done.
> Allow native services quicklinks to be exported for each component
> --
>
> Key: YARN-5993
> URL: https://issues.apache.org/jira/browse/YARN-5993
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
> Attachments: YARN-5993-yarn-native-services.001.patch
>
>
> The quicklinks export capability changed in switching from the agent provider
> to the docker provider, and currently the docker provider only allows one
> component to export links. We should improve this capability to be more in
> line with the previous agent capability.
--
This message was sent by Atlassian
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755889#comment-15755889
]
Gour Saha commented on YARN-5993:
-
[~billie.rinaldi] Please fix the following findbug warning and the 6 new
checkstyle issues as per the jenkins report.
1 findbug warning -
At DockerProviderService.java:line 368
{code}
org.apache.slider.providers.docker.DockerProviderService.publishExportGroups(String,
String, String, String, List) makes inefficient use of keySet iterator instead
of entrySet iterator
{code}
h6.
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/core/registry/docstore/ExportEntry.java
equals and hashcode methods:
{code}
return containerId.equals(that.containerId);
return containerId.hashCode();
{code}
Please add null check for containerId in both the methods.
h6.
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/core/registry/docstore/PublishedExports.java
{code}
public Map sortedEntries() {
TreeMap sortedEntries = new TreeMap<>();
{code}
Change sortedEntries type from TreeMap to Map
h6.
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/providers/ProviderUtils.java
Remove Iterator import.
{code}
public synchronized void publishExportGroup(
Map exportGroup,
StateAccessForProviders amState, String groupName) {
{code}
Why is this method synchronized now?
Method publishExportGroup:
{code}
TreeSet values = new TreeSet<>();
{code}
We can change values type from TreeSet to Set.
h6.
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/providers/docker/DockerProviderService.java
Method notifyContainerCompleted:
{code}
for (Set entrySet : exportMap.values()) {
{code}
Can you rename entrySet to entries or exportEntries or something more
appropriate to avoid confusion to the reader who might think why it is
exportMap.values() and not exportMap.entrySet()?
Method publishExportGroups:
1.
{code}
replaceTokens.put(String.format(IP_KEY_FORMAT, roleNameKey), ips.get(0));
.
.
replaceTokens.put(String.format(IP_KEY_FORMAT, roleGroupKey),
ips.get(0));
{code}
Since we are discarding all but the first IP, I am thinking that we should
probably introduce a IPS_KEY_FORMAT of $\{%s_IPS\} and replace it with a comma
separated list of IPs when there are more than one IP for a single container.
This gives application owners a placeholder to fetch all the IPs as well (if
they want to).
2.
Question:
Just to confirm, if roleNameKey is null then there is no roleGroupKey, right?
3.
{code}
if (value.contains(VARIABLE_INDICATOR)) {
// not all variables have been substituted, so do not export
continue;
}
{code}
I think we should not block here, since it will help application owners to know
which variables were not substituted, rather than thinking that exports don’t
work at all. What do you think?
Method getExportSet:
{code}
protected Set getExportSet(String key) {
{code}
Can we rename this method to getExportEntries(String key) ?
h6.
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/java/org/apache/slider/server/appmaster/web/view/IndexBlock.java
Method enumeratePublishedExports:
{code}
if (entry.getValue().size() > 0) {
{code}
Check entry.getValue() for null
> Allow native services quicklinks to be exported for each component
> --
>
> Key: YARN-5993
> URL: https://issues.apache.org/jira/browse/YARN-5993
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
> Attachments: YARN-5993-yarn-native-services.001.patch
>
>
> The quicklinks export capability changed in switching from the agent provider
> to the docker provider, and currently the docker provider only allows one
> component to export links. We should improve this capability to be more in
> line with the previous agent capability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5993) Allow native services quicklinks to be exported for each component
[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752132#comment-15752132
]
Hadoop QA commented on YARN-5993:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
39s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
9s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
in yarn-native-services has 262 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
32s{color} | {color:red} hadoop-yarn-slider-core in yarn-native-services
failed. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 6 new + 169 unchanged - 7 fixed = 175 total (was 176)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
16s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
generated 1 new + 261 unchanged - 1 fixed = 262 total (was 262) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
31s{color} | {color:red} hadoop-yarn-slider-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
|
| |
org.apache.slider.providers.docker.DockerProviderService.publishExportGroups(String,
String, String, String, List) makes inefficient use of keySet iterator instead
of entrySet iterator At DockerProviderService.java:makes inefficient use of
keySet iterator instead of entrySet iterator At
DockerProviderService.java:[line 368] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | YARN-5993 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843445/YARN-5993-yarn-native-services.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 57b2824b4aef 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64
