[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861720#comment-16861720 ] Prabhu Joseph commented on HADOOP-16354: Thanks [~eyang]. > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch, HADOOP-16354-004.patch, HADOOP-16354-005.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861598#comment-16861598 ] Hudson commented on HADOOP-16354: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16728 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16728/]) HADOOP-16354. Enable AuthFilter as default for WebHDFS.(eyang: rev 4ea6c2f457496461afc63f38ef4cef3ab0efce49) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilterInitializer.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilter.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authentication/server/TestProxyUserAuthenticationFilter.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilter.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestAuthFilter.java > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch, HADOOP-16354-004.patch, HADOOP-16354-005.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861569#comment-16861569 ] Eric Yang commented on HADOOP-16354: +1 for patch 005. > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch, HADOOP-16354-004.patch, HADOOP-16354-005.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860776#comment-16860776 ] Prabhu Joseph commented on HADOOP-16354: Failing testcases are not related and it works fine on local. > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch, HADOOP-16354-004.patch, HADOOP-16354-005.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860757#comment-16860757
]
Hadoop QA commented on HADOOP-16354:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
54s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
19s{color} | {color:green} root: The patch generated 0 new + 62 unchanged - 12
fixed = 62 total (was 74) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 44s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
48s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 6s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}238m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.qjournal.client.TestQJMWithFaults |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16354 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971414/HADOOP-16354-005.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5276f34cfbcb 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860581#comment-16860581
]
Prabhu Joseph commented on HADOOP-16354:
[~eyang] Missed to test with doas, was testing with all other combinations.
With doas set for webhdfs requests without delegation token, impersonation
logic is called twice - one at {{ProxyUserAuthenticationFilter}} and then at
{{JspHelper#getUgi}}. Have ignored calling impersonation if the remote user is
same as doas user.
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch,
> HADOOP-16354-003.patch, HADOOP-16354-004.patch, HADOOP-16354-005.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860412#comment-16860412
]
Hadoop QA commented on HADOOP-16354:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} root: The patch generated 0 new + 54 unchanged - 12
fixed = 54 total (was 66) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 48s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 45s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 13s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}221m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.security.authentication.server.TestProxyUserAuthenticationFilter |
| | hadoop.hdfs.server.namenode.TestEditLogAutoroll |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16354 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971366/HADOOP-16354-004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 2c91e3640a09 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar
18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860359#comment-16860359
]
Eric Yang commented on HADOOP-16354:
[~Prabhu Joseph] Thank you for patch 004, it is closer to what we need, but I
can't get it to work with lower case doas=, even though the patch seems to
convert to lower case for doas.
{code}
[hdfs@eyang-1 hadoop-3.3.0-SNAPSHOT]$ curl --negotiate -u : "http://`hostname
-f`:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs=eyang"
{"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed
to obtain user group information:
org.apache.hadoop.security.authorize.AuthorizationException: User: eyang is not
allowed to impersonate eyang"}}{code}
When using doAs, then it works as expected:
{code}
[hdfs@eyang-1 hadoop-3.3.0-SNAPSHOT]$ curl --negotiate -u : "http://`hostname
-f`:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs=eyang"
{"Token":{"urlString":"HQAFZXlhbmcEaGRmcwCKAWtDUn5oigFrZ18CaAECFJ6Dq3M5Slq_QhusB9mHwZcj8axREldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuMTExLjE3OjkwMDAA"}}
[eyang@eyang-1 root]$ curl -L "http://`hostname
-f`:50070/webhdfs/v1/user/hdfs/README.txt?op=GETFILESTATUS=HQAFZXlhbmcEaGRmcwCKAWtDUn5oigFrZ18CaAECFJ6Dq3M5Slq_QhusB9mHwZcj8axREldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuMTExLjE3OjkwMDAA"
{"RemoteException":{"exception":"AccessControlException","javaClassName":"org.apache.hadoop.security.AccessControlException","message":"Permission
denied: user=eyang, access=EXECUTE,
inode=\"/user/hdfs\":hdfs:hdfs:drwx--"}}
{code}
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch,
> HADOOP-16354-003.patch, HADOOP-16354-004.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860281#comment-16860281 ] Prabhu Joseph commented on HADOOP-16354: Thanks [~eyang] for reviewing. Have modified AuthFilter to extend ProxyUserAuthenticationFilter so that doas support is provided for NameNode UI + WebHdfs. Both accepts case insensitive doas flag. Have tested both 2.1 and 2.2 test cases, it works fine. > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch, HADOOP-16354-004.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860140#comment-16860140
]
Eric Yang commented on HADOOP-16354:
[~Prabhu Joseph] Test case 2 is mixed for getting delegation token, and
accessing via knox gateway. However, doAs flag is missing when requesting
delegation token. Hence, the token returned from webhdfs is owned by Knox user
instead of ambari-qa.
We can refine the test into two separate tests.
h2. 2.1 Knox obtain delegation token for end user for cross knox distcp
The test must be written as:
{code}
[knox@pjosephdocker-1 hadoop]$ curl --negotiate -u :
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs=ambari-qa;
{"Token":{"urlString":"hash of delegation token for ambari-qa user"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS=hash
of delegation token for ambari-qa user"
{code}
The key difference is in obtaining GETDELEGATIONTOKEN operation and doAs flag
needs to work together for knox to obtain a valid toke for the end user. In
the past, we allow doas= and also doAs=, this was a case insensitive flag.
h2. 2.2 Normal operation to get delegation token as end user for distcp
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl --negotiate -u :
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs;
{"Token":{"urlString":"IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/knox?op=GETFILESTATUS=IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA;
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":1394411,"group":"hadoop","length":0,"modificationTime":1559980208213,"owner":"knox","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
{code}
The test case 2.1 must work for in AuthFilter regardless if
ProxyUserAuthenticationFilter or AuthenticationFilter is configured to maintain
backward compatibility.
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch,
> HADOOP-16354-003.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859147#comment-16859147
]
Prabhu Joseph commented on HADOOP-16354:
[~eyang] WebHdfs provides doas support indirectly through {{JspHelper#getUGI}}.
{code}
if (doAsUserFromQuery != null) {
// create and attempt to authorize a proxy user
ugi = UserGroupInformation.createProxyUser(doAsUserFromQuery, ugi);
ProxyUsers.authorize(ugi, getRemoteAddr(request));
}
{code}
All the three scenarios in
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Proxy_Users
works fine.
*Test Cases:*
{code}
Test Case 1:
**
A proxy request using Kerberos SPNEGO when security is on (knox impersonates
ambari-qa)
[knox@pjosephdocker-1 hadoop]$ curl --negotiate -u :
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/ambari-qa?doas=ambari-qa=GETFILESTATUS;
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":14,"fileId":16388,"group":"hdfs","length":0,"modificationTime":1559557387372,"owner":"ambari-qa","pathSuffix":"","permission":"770","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
2019-06-08 07:58:09,649 DEBUG common.JspHelper (JspHelper.java:getUGI(173)) -
getUGI is returning: ambari-qa
Test Case 2:
**
A proxy request using Hadoop delegation token when security is on:
[knox@pjosephdocker-1 hadoop]$ curl --negotiate -u :
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs;
{"Token":{"urlString":"IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"}}
[ambari-qa@pjosephdocker-1 ~]$ curl
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/knox?op=GETFILESTATUS=IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA;
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":1394411,"group":"hadoop","length":0,"modificationTime":1559980208213,"owner":"knox","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
2019-06-08 08:00:32,679 DEBUG common.JspHelper (JspHelper.java:getUGI(173)) -
getUGI is returning: knox
{code}
Have initially using {{AuthFIlter}} to extend {{ProxyUserAuthenticationFilter}}
but left it as the {{JspHelper#getUG}}I already does.
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch,
> HADOOP-16354-003.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859129#comment-16859129
]
Hadoop QA commented on HADOOP-16354:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 5s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 54 unchanged - 12 fixed = 54 total (was 66) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 16s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 12s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16354 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971225/HADOOP-16354-003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 10b8d4ecabec 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar
18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 46b23c1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16306/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16306/testReport/ |
| Max. process+thread count | 2898 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859120#comment-16859120 ] Eric Yang commented on HADOOP-16354: If AuthFilter is extended from AuthenticationFilter, then webhdfs doesn't honor ?doAs= flag. This breaks compatibility: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Proxy_Users When user access webhdfs via Knox. Knox credential would be used. I think AuthFilter should extend from ProxyUserAuthenticationFilter to ensure that doAs flag is honored. AuthFilter only ignores doAs flag when DT is in use. > Enable AuthFilter as default for WebHdfs > > > Key: HADOOP-16354 > URL: https://issues.apache.org/jira/browse/HADOOP-16354 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, > HADOOP-16354-003.patch > > > HADOOP-16314 provides an generic option to configure > ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all > the services. If this is not configured, AuthenticationFIlter is used for > NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so > that it is backward compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859099#comment-16859099
]
Prabhu Joseph commented on HADOOP-16354:
[~eyang] Thanks for reviewing.
1. Have removed setting {{simple.anonymous.allowed}} to true by default.
2. Before HADOOP-16314, The default Filter is {{AuthFilter}} for WebHdfs and
{{AuthenticationFilter}} for NameNode UI. WebHdfs can provide delegation token
support only when {{AuthFilter}} + {{UserProvider}} Injector (which calls
{{JspHelper#getUGI}}) is configured. Have retained the same default of
{{AuthFIlter}} for WebHdfs for backward compatibility. Users can configure
{{ProxyUserAuthenticationFilterInitializer}} if required which will exclude
{{AuthFIlter}}.
3. The Default {{AuthFilter}} fixes Distcp with WebHdfs as well. (HADOOP-16356).
MapReduce JobClient fetches delegation token from WebHdfs. This works with
valid kerberos ticket. Any {{AuthenticationFilter}}
({{ProxyUserAuthenticationFilter}} or {{AuthFilter}}) which does kerberos
authentication will be able to provide a token.
{code:java}
curl --negotiate -u :
"http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN=hdfs;
{"Token":{"urlString":"IAAEa25veARoZGZzAIoBayNUC66KAWtHYI-ujgGxjgFnFKE9HVj_mxbfJd2lxzNGMHRDx_wVEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"}}
{code}
But when mapreduce tasks uses the token in subsequent call, WebHdfs has to be
configured with {{AuthFilter}} to perform delegation token authentication. Both
{{ProxyUserAuthenticationFilter}} and {{AuthenticationFilter}} will fail with
"Authentication Required" as it expects only kerberos authentication.
{code:java}
curl
'http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/services/sync/yarn-ats?op=GETFILESTATUS=IAAEa25veARoZGZzAIoBay16h0mKAWtRhwtJjgG1jgF6FHXhPdw7C4nPpM7-P97b_BbPRr-9EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA'
< Authentication required >
{code}
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch,
> HADOOP-16354-003.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859028#comment-16859028
]
Eric Yang commented on HADOOP-16354:
{code}
+// if not set, enable anonymous for pseudo authentication
+if (filterConfig.get(PseudoAuthenticationHandler.ANONYMOUS_ALLOWED)
+== null) {
+ filterConfig.put(PseudoAuthenticationHandler.ANONYMOUS_ALLOWED, "true");
+}
{code}
Patch 002 default anonymous to allow to connect to webhdfs even when user
configured to use hadoop.http.authentication.type != simple. This is a
dangerous default that may keep webhdfs open to everyone if system admin did
not know to set hadoop.http.authentication.simple.anonymous.allowed = false.
In the code, it only allows AuthFilter to be configured, when
ProxyUserAuthenticationFilter is not configured. I don't think I full
understand the reasoning for this exclusion. Could you explain again? Thanks
> Enable AuthFilter as default for WebHdfs
>
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch
>
>
> HADOOP-16314 provides an generic option to configure
> ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all
> the services. If this is not configured, AuthenticationFIlter is used for
> NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so
> that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858987#comment-16858987
]
Hadoop QA commented on HADOOP-16354:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 55 unchanged - 12 fixed = 55 total (was 67) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 49s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 26s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}154m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestCrcCorruption |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestMultipleNNPortQOP |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16354 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971196/HADOOP-16354-002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6124348d4ef7 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec
10 13:20:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 4e38daf |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16305/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16305/testReport/ |
| Max. process+thread count | 2890 (vs. ulimit of
[jira] [Commented] (HADOOP-16354) Enable AuthFilter as default for WebHdfs
[
https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858837#comment-16858837
]
Hadoop QA commented on HADOOP-16354:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
35s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
35s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 49 unchanged - 10 fixed = 49 total (was 59) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
36s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
14s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 37s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16354 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971189/HADOOP-16354-001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0ec3f6cb2485 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec
10 13:20:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8547957 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16304/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16304/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16304/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| mvnsite |
