[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246138#comment-16246138 ] Xiao Chen commented on HDFS-12052: -- Committed this to branch-2, after resolving minor conflicts. Ran the test locally before pushing. Thanks again > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1, 2.10.0 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245286#comment-16245286 ] Zoran Dimitrijevic commented on HDFS-12052: --- Thank you! > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245284#comment-16245284 ] Xiao Chen commented on HDFS-12052: -- Aha, thank you for the response. I skimmed through the comments and assumed this wasn't committed to branch-2 because of conflicts. Sorry for the wrong assumption. Just tried backporting, and the conflict is pretty minor. I'll just go ahead and cherry-pick this to branch-2 if no objections. Thanks! > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245268#comment-16245268 ] Zoran Dimitrijevic commented on HDFS-12052: --- Sure. But I think it was exactly the same fix as for 3+. I did it for Altiscale 2.7+ branch. What do you want me to do? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244497#comment-16244497 ] Xiao Chen commented on HDFS-12052: -- Thanks [~3opan] for reporting and fixing this issue! Would you be interested to provide a branch-2 fix as well? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082754#comment-16082754 ] Hudson commented on HDFS-12052: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11988 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11988/]) HDFS-12052. Set SWEBHDFS delegation token kind when ssl is enabled in (jzhuge: rev 12c8fdceaf263425661169cba25402df89d444c1) * (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSAuthenticationFilter.java * (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSServerWebServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/HttpFSKerberosAuthenticationHandlerForTesting.java > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081584#comment-16081584 ] John Zhuge commented on HDFS-12052: --- Thanks [~3opan]. Will commit tomorrow morning so that others have a chance to comment or review. > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081560#comment-16081560 ] Zoran Dimitrijevic commented on HDFS-12052: --- [~jzhuge] thanks. Removed unnecessary import and it seems it's finally all good. Please submit the patch when you have time. Cheers. > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081552#comment-16081552
]
Hadoop QA commented on HDFS-12052:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch
generated 0 new + 53 unchanged - 9 fixed = 53 total (was 62) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
20s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12052 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876550/HDFS-12052.07.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 052717cd9ef9 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f1efa14 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20221/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U:
hadoop-hdfs-project/hadoop-hdfs-httpfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20221/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran Dimitrijevic
>Assignee: Zoran Dimitrijevic
> Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch,
> HDFS-12052.02.patch, HDFS-12052.03.patch,
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081472#comment-16081472 ] Zoran Dimitrijevic commented on HDFS-12052: --- I run dev-support/bin/test-patch --dirty-workspace --plugins=checkstyle,maven HDFS-12052.06.patch and it was +1. So, how do I see which new style offense still exists in this patch? | Vote | Subsystem | Runtime | Comment | +1 |mvninstall | 9m 48s | trunk passed | +1 |checkstyle | 0m 13s | trunk passed | +1 |checkstyle | 0m 9s | the patch passed | || 10m 21s | > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081494#comment-16081494
]
John Zhuge commented on HDFS-12052:
---
Clicked the link
https://builds.apache.org/job/PreCommit-HDFS-Build/20219/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt:
{noformat}
./hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSAuthenticationFilter.java:22:import
org.apache.hadoop.fs.http.server.HttpFSServerWebServer;:1: Redundant import
from the same package - org.apache.hadoop.fs.http.server.HttpFSServerWebServer.
[RedundantImport]
{noformat}
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran Dimitrijevic
>Assignee: Zoran Dimitrijevic
> Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch,
> HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch,
> HDFS-12052.05.patch, HDFS-12052.06.patch
>
>
> When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation
> tokens.
> Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of
> whether ssl is enabled or not. If clients directly connect to renew tokens
> (for example, hdfs dfs) all works because httpfs doesn't check whether token
> kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to
> renew the token for the job (for example, when running hadoop distcp). Since
> DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and
> fails.
> I've tested a simple patch which I'll upload to this jira, and it fixes this
> issue (hadoop distcp works).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081451#comment-16081451 ] John Zhuge commented on HDFS-12052: --- +1 LGTM pending the last minor checkstyle issue > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081409#comment-16081409
]
Hadoop QA commented on HDFS-12052:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-httpfs: The
patch generated 1 new + 52 unchanged - 9 fixed = 53 total (was 61) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
17s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12052 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876534/HDFS-12052.06.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux baafe598894f 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5496a34 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20219/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20219/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U:
hadoop-hdfs-project/hadoop-hdfs-httpfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20219/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081394#comment-16081394 ] John Zhuge commented on HDFS-12052: --- Yetus only reports checkstyle issues on the modified lines. I am also very eager to make everything I touch as pretty as possible. Unfortunately too many gratuitous changes are not manageable when you have to maintain dozens of branches and do backports constantly, often in different commit orders. Clean backports are very much appreciated. Thank you [~3opan] for removing the style fixes. That said, style fixes on or very close to the modified lines are ok. Please do file refactor JIRAs where we can anticipate conflicts and resolve them just once. > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081276#comment-16081276
]
John Zhuge commented on HDFS-12052:
---
Sorry for the confusion. The correct property name is {{httpfs.ssl.enabled}}.
The property names are documented in doc and httpfs-default.xml. Environment
variables are deprecated.
HDFS-10860 made mistakes in using both hadoop.httpfs.ssl.enabled and
httpfs.ssl.enabled. Filed HDFS-12114 to fix.
For your patch, please use the following code:
{code:java}
conf.setBoolean(HttpFSServerWebServer.SSL_ENABLED_KEY, true);
{code}
You'd have to make {{HttpFSServerWebServer.SSL_ENABLED_KEY}} package private.
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran Dimitrijevic
>Assignee: Zoran Dimitrijevic
> Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch,
> HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch,
> HDFS-12052.05.patch
>
>
> When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation
> tokens.
> Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of
> whether ssl is enabled or not. If clients directly connect to renew tokens
> (for example, hdfs dfs) all works because httpfs doesn't check whether token
> kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to
> renew the token for the job (for example, when running hadoop distcp). Since
> DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and
> fails.
> I've tested a simple patch which I'll upload to this jira, and it fixes this
> issue (hadoop distcp works).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081240#comment-16081240
]
Ravi Prakash commented on HDFS-12052:
-
I tried understanding the whole sequence of configurations set in environment
variables, xml files, properties being passed around between {{HttpFS*}} and
{{HttpServer2}} and {{AuthenticationFilter}}, and threw up my hands.
It makes sense to me that when HttpFS is configured with SSL, the delegation
tokens returned by the server should be SWEBHDFS.
Just one change, could you please use {{HttpFSServerWebServer.SSL_ENABLED_KEY}}
instead of the hard coded string {{"httpfs.ssl.enabled"}}?
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran Dimitrijevic
>Assignee: Zoran Dimitrijevic
> Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch,
> HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch,
> HDFS-12052.05.patch
>
>
> When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation
> tokens.
> Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of
> whether ssl is enabled or not. If clients directly connect to renew tokens
> (for example, hdfs dfs) all works because httpfs doesn't check whether token
> kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to
> renew the token for the job (for example, when running hadoop distcp). Since
> DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and
> fails.
> I've tested a simple patch which I'll upload to this jira, and it fixes this
> issue (hadoop distcp works).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[
https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081044#comment-16081044
]
Hadoop QA commented on HDFS-12052:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch
generated 0 new + 7 unchanged - 54 fixed = 7 total (was 61) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
24s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12052 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876483/HDFS-12052.05.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f7537a72e3c1 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 09653ea |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20214/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U:
hadoop-hdfs-project/hadoop-hdfs-httpfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20214/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
>
>
> Key: HDFS-12052
> URL: https://issues.apache.org/jira/browse/HDFS-12052
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs, webhdfs
>Affects Versions: 2.7.3, 3.0.0-alpha3
>Reporter: Zoran Dimitrijevic
>Assignee: Zoran Dimitrijevic
> Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch,
> HDFS-12052.02.patch, HDFS-12052.03.patch,
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080956#comment-16080956 ] Allen Wittenauer commented on HDFS-12052: - bq. They add noise to the code review and will complicate backport. I don't think this is going to be easy to backport anyway, given the shifts in code between 2 and 3. Personally, I'd prefer we fix style issues here anyway so that future commits don't have these issues. It's just a guideline to not fix style issues. The much bigger issue is this one: bq. Do we want to use httpfs.ssl.enabled or hadoop.httpfs.ssl.enabled ? This is a much bigger problem. It looks like the shell profile code is wrong, but [~jzhuge] would know for sure. We should probably fix this at the same time while we are here. > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080797#comment-16080797 ] Zoran Dimitrijevic commented on HDFS-12052: --- Sure [~jzhuge]. I had to reformat the tests which are then detected as new code by style checks - I'll remove all style fixes unrelated to test refactoring. You can then review the new patch which will contain these fixes so that we don't waste this opportunity to clean up the style as well? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
