[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039737#comment-17039737
]
Hadoop QA commented on YARN-9208:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} YARN-9208 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958513/YARN-9208-004.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25537/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Distributed shell allow LocalResourceVisibility to be specified
> ---
>
> Key: YARN-9208
> URL: https://issues.apache.org/jira/browse/YARN-9208
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin Chundatt
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: YARN-9208-001.patch, YARN-9208-002.patch,
> YARN-9208-003.patch, YARN-9208-004.patch
>
>
> YARN-9008 add feature to add list of files to be localized.
> Would be great to have Visibility type too. Allows testing of PRIVATE and
> PUBLIC type too
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804673#comment-16804673 ] Prabhu Joseph commented on YARN-9208: - Hi [~bibinchundatt], Can you review the latest patch when you get time. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch, > YARN-9208-003.patch, YARN-9208-004.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776669#comment-16776669 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Can you check the latst patch when you get time. Thanks. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch, > YARN-9208-003.patch, YARN-9208-004.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772683#comment-16772683
]
Prabhu Joseph commented on YARN-9208:
-
[~bibinchundatt] Have changed the pattern into
{{(PUBLIC=FileName1,FileName2,,),(PRIVATE=FileName3,FileName4,,),,}}. If only a
PRIVATE file hdfs:/tmp/a is present - the pattern will be
(PRIVATE=hdfs:/tmp/a). Can you review the same.
> Distributed shell allow LocalResourceVisibility to be specified
> ---
>
> Key: YARN-9208
> URL: https://issues.apache.org/jira/browse/YARN-9208
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin A Chundatt
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: YARN-9208-001.patch, YARN-9208-002.patch,
> YARN-9208-003.patch, YARN-9208-004.patch
>
>
> YARN-9008 add feature to add list of files to be localized.
> Would be great to have Visibility type too. Allows testing of PRIVATE and
> PUBLIC type too
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766879#comment-16766879
]
Hadoop QA commented on YARN-9208:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
37s{color} | {color:green} hadoop-yarn-applications-distributedshell in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958513/YARN-9208-004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux daccc8c33c2b 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 917ac9f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23394/testReport/ |
| Max. process+thread count | 642 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23394/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766481#comment-16766481
]
Hadoop QA commented on YARN-9208:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 46s{color}
| {color:red} hadoop-yarn-applications-distributedshell in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.yarn.applications.distributedshell.TestDSWithMultipleNodeManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958444/YARN-9208-003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 3696b268c2e6 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 7806403 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/23390/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23390/testReport/ |
| Max. process+thread count | 583 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765691#comment-16765691 ] Prabhu Joseph commented on YARN-9208: - Yes [~bibinchundatt], Working on it, will update. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765685#comment-16765685 ] Bibin A Chundatt commented on YARN-9208: [~Prabhu Joseph] On second thought , Pattern fileName1=visibilityType,fileName2=visibilityType usability is good only for single file. If we have to give multiple files of visibility type public,application then too many times we are giving visibility Type. Can we optimize this part ? > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764750#comment-16764750 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Can you review the latest patch, the test case timeout is fixed in YARN-9231 > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757168#comment-16757168
]
Hadoop QA commented on YARN-9208:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 38s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 32s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 5s{color}
| {color:red} hadoop-yarn-applications-distributedshell in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957071/YARN-9208-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 9bacae4163a4 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 71c49fa |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/23236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23236/testReport/ |
| Max. process+thread count | 617 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
|
| Console output |
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757119#comment-16757119 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Attached new patch with suggested changes. 1. Used Enum valueOf instead of switch case. 2. Added visibility type to localized_files with format fileName1=visibilityType,fileName2=visibilityType,, . Have used lastIndexOf to split fileName and visibilityType and so fileName can have =. bq. Does file scheme allow PUBLIC definition ?? File Scheme won't have Public type as the Hdfs staging directory (user.home/DistributedShell) is Private. The local file is copied to hdfs staging directory and then isPublic is checked to avoid IOException(File is not publicly accessible and as such cannot be part of the public cache) from FsDownload. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch, YARN-9208-002.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756927#comment-16756927 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Thanks for the review, working on it. Will update you. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756890#comment-16756890
]
Bibin A Chundatt commented on YARN-9208:
[~Prabhu Joseph] Sorry for the delay.
Few comments
{code:java}
764 case "true": return LocalResourceVisibility.PUBLIC;
765 case "false": return LocalResourceVisibility.PRIVATE;
{code}
Better to give visibility type as argument (public,private,application) for
files
{code:java}
744 // Default Visibilty is set to APPLICATION type
745 int size = fileNames.size();
746 if (size == visibilities.size()) {
747 for (int i=0; ivisibilityType,filename2visibilityType" . Rename key
(localized_files) if required. Limitation of "size == visibilities.size()" we
could avoid.
{quote}The Client determines the visibilities (PRIVATE, PUBLIC and default
APPLICATION) based upon the permission of the input file. Also supports input
files from Hdfs.
{quote}
Does file scheme allow PUBLIC definition ??
> Distributed shell allow LocalResourceVisibility to be specified
> ---
>
> Key: YARN-9208
> URL: https://issues.apache.org/jira/browse/YARN-9208
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin A Chundatt
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: YARN-9208-001.patch
>
>
> YARN-9008 add feature to add list of files to be localized.
> Would be great to have Visibility type too. Allows testing of PRIVATE and
> PUBLIC type too
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755705#comment-16755705 ] Prabhu Joseph commented on YARN-9208: - Test cases Timed out will be fixed as part of YARN-9231 > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755328#comment-16755328
]
Hadoop QA commented on YARN-9208:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
2m 7s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 3s{color}
| {color:red} hadoop-yarn-applications-distributedshell in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956768/YARN-9208-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux acb05598d740 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 04105bb |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/23223/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23223/testReport/ |
| Max. process+thread count | 632 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
|
| Console output |
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755282#comment-16755282 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Have updated the patch with - The Client determines the visibilities (PRIVATE, PUBLIC and default APPLICATION) based upon the permission of the input file. Also supports input files from Hdfs. Can you review the patch when you get some time. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > Attachments: YARN-9208-001.patch > > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[ https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750038#comment-16750038 ] Prabhu Joseph commented on YARN-9208: - [~bibinchundatt] Have two options to set the visibilities for localized files - Client itself determines based on the permission of file or User passes the visibilities for each file. Can you review which one will be fine. > Distributed shell allow LocalResourceVisibility to be specified > --- > > Key: YARN-9208 > URL: https://issues.apache.org/jira/browse/YARN-9208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Prabhu Joseph >Priority: Minor > > YARN-9008 add feature to add list of files to be localized. > Would be great to have Visibility type too. Allows testing of PRIVATE and > PUBLIC type too -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
