[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677123#comment-16677123
]
Hudson commented on HDFS-14051:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15373 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/15373/])
HDFS-14051. Refactor NameNodeHttpServer#initWebHdfs to specify local (inigoiri:
rev f0ce072934515d39e9bf61744058ecad3393291e)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterHttpServer.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14051.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677108#comment-16677108
]
CR Hota commented on HDFS-14051:
[~elgoiri]
Thanks for the quick review and commit.
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14051.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677094#comment-16677094
]
Íñigo Goiri commented on HDFS-14051:
+1 on [^HDFS-14051.0.patch].
This is just a refactor and no change in functionality so no need for unit
tests.
Committing to trunk.
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14051.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677012#comment-16677012
]
CR Hota commented on HDFS-14051:
[~elgoiri]
Please take a look at the patch. This patch allows to pass a keytab while
initializing webhdfs. Once we have kerberos patch ready, I can add that in
routerhttpserver instead of "null" being passed as the keytab.
The failed unit test succeeded in my local set-up. Seems like a transient
issue.
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 71.431
s - in org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0
[INFO]
{code}
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14051.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676085#comment-16676085
]
Hadoop QA commented on HDFS-14051:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 35s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 9s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
2s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}201m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestEncryptionZonesWithKMS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14051 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12946986/HDFS-14051.0.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f9177ab6fcf0 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f3f5e7a |
| maven | version: Apache Maven 3.3.9 |
| Default Java |
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675808#comment-16675808
]
Hadoop QA commented on HDFS-14051:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} HDFS-13532 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
4s{color} | {color:green} HDFS-13532 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
20s{color} | {color:green} HDFS-13532 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} HDFS-13532 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} HDFS-13532 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} HDFS-13532 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} HDFS-13532 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 1s{color} | {color:orange} hadoop-hdfs-project: The patch generated 3 new +
9 unchanged - 0 fixed = 12 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 49s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 12s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 18s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}206m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDistributedFileSystem |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | hadoop.hdfs.server.federation.router.TestRBFConfigFields |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14051 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12946948/HDFS-14051-HDFS-13532.0.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6423f3f75bb1 3.13.0-153-generic #203-Ubuntu SMP Thu Jun
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675687#comment-16675687
]
CR Hota commented on HDFS-14051:
[~elgoiri] Thanks for the suggestion. Will do the change separately, this
should be done on top of trunk then.
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14051-HDFS-13532.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675601#comment-16675601
]
Íñigo Goiri commented on HDFS-14051:
Thanks [~crh] for the patch.
I would just do the change in the Namenode side.
In addition, I would change getAuthFilterParams() there to make it directly
public and pass a parameter for the keytab; then we can just use it from the
Router later.
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14051-HDFS-13532.0.patch
>
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14051) Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
[
https://issues.apache.org/jira/browse/HDFS-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673780#comment-16673780
]
Íñigo Goiri commented on HDFS-14051:
This is needed by HDFS-12284 as we need to specify the keytab for the Router.
> Refactor NameNodeHttpServer#initWebHdfs to specify local keytab
> ---
>
> Key: HDFS-14051
> URL: https://issues.apache.org/jira/browse/HDFS-14051
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
>
> We use {{NameNodeHttpServer#initWebHdfs}} from {{RouterHttpServer}}.
> However, this relies on {{NameNodeHttpServer#getAuthFilterParams()}} which
> uses:
> {code}
> String httpKeytab = conf.get(DFSUtil.getSpnegoKeytabKey(conf,
> DFSConfigKeys.DFS_NAMENODE_KEYTAB_FILE_KEY));
> {code}
> We should refactor this to be able to specify the keytab file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
