[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626749#comment-15626749 ] Haibo Chen commented on YARN-5001: -- Thanks [~jlowe] for your kind reviews! > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: yarn5001.001.patch, yarn5001.002.patch, > yarn5001.003.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626614#comment-15626614 ] Hudson commented on YARN-5001: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10746 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10746/]) YARN-5001. Aggregated Logs root directory is created with wrong group if (jlowe: rev 76893a41003d57d94eb1a5f486010815266046af) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: yarn5001.001.patch, yarn5001.002.patch, > yarn5001.003.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626571#comment-15626571 ] Jason Lowe commented on YARN-5001: -- +1 lgtm. Committing this. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Attachments: yarn5001.001.patch, yarn5001.002.patch, > yarn5001.003.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[
https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623316#comment-15623316
]
Hadoop QA commented on YARN-5001:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
54s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5001 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12836210/yarn5001.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5ae9fbc2a76c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a9d68d2 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13701/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13701/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Aggregated Logs root directory is created with wrong group if nonexistent
> --
>
> Key: YARN-5001
> URL: https://issues.apache.org/jira/browse/YARN-5001
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation, nodemanager, security
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Labels: oct16-easy
>
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623085#comment-15623085 ] Jason Lowe commented on YARN-5001: -- Thanks for updating the patch! Looks good with one nit: we don't need two variables to track whether we have a primary group. hasPrimaryGroup is simply primaryGroup != null, and it's simpler to just check it directly than track it in a boolean. May be worth logging a warning in the case there is no primary group that the group permissions may need admin intervention to fix the issue with visibility for the JHS. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Attachments: yarn5001.001.patch, yarn5001.002.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[
https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622689#comment-15622689
]
Hadoop QA commented on YARN-5001:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
3s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5001 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12836173/yarn5001.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 86e62be62748 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f646fe3 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13697/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13697/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Aggregated Logs root directory is created with wrong group if nonexistent
> --
>
> Key: YARN-5001
> URL: https://issues.apache.org/jira/browse/YARN-5001
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation, nodemanager, security
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Labels: oct16-easy
>
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622484#comment-15622484 ] Haibo Chen commented on YARN-5001: -- Thanks [~jlowe] for your review. Will upload a new patch address it. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Attachments: yarn5001.001.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[
https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613343#comment-15613343
]
Hadoop QA commented on YARN-5001:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
8s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5001 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800906/yarn5001.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b9ec1aeaf16f 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / dd4ed6a |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13587/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13587/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Aggregated Logs root directory is created with wrong group if nonexistent
> --
>
> Key: YARN-5001
> URL: https://issues.apache.org/jira/browse/YARN-5001
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation, nodemanager, security
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Labels: oct16-easy
>
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613068#comment-15613068 ] Jason Lowe commented on YARN-5001: -- What if the user has no primary group? In a secure environment the login user may not map to an OS user and could have no groups. If that's the case then this change would make it so no logs are aggregated at all due to the IOException thrown by getPrimaryGroupName. Probably would be prudent to only do the setOwner if we actually have a group name to use, otherwise this change behaves worse than it does today if we fail to find a group on the login user. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation, nodemanager, security >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Labels: oct16-easy > Attachments: yarn5001.001.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289882#comment-15289882 ] Ray Chiang commented on YARN-5001: -- +1 (nonbinding). Looks like a simple enough change. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: yarn5001.001.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272908#comment-15272908 ] Haibo Chen commented on YARN-5001: -- The fix will explicitly set the group of /tmp/logs to the group that the user Node Manager is running as belongs to. This will not work if the group of the user JHS is running as is not the same as that of Node Manager. > Aggregated Logs root directory is created with wrong group if nonexistent > -- > > Key: YARN-5001 > URL: https://issues.apache.org/jira/browse/YARN-5001 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.7.0 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: yarn5001.001.patch > > > The directory /tmp/logs, where the aggregated logs go, is supposed to be read > by JHS. But if it is not created beforehand, it will be created by > NodeManager with the group being the superuser set in HDFS. Files created > under this directory will then inherit the supergroup as their groups. This > leads to JHS to fail to read the container logs files under that directory if > JHS is not running as a user that belongs to superuser. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
[
https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259203#comment-15259203
]
Hadoop QA commented on YARN-5001:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 2s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_92. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 33s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800906/yarn5001.001.patch |
| JIRA Issue | YARN-5001 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 6d2eb196ba39 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 6be22dd |
| Default Java | 1.7.0_95 |
| Multi-JDK versions |
