[
https://issues.apache.org/jira/browse/HDFS-15860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17640583#comment-17640583
]
Zbigniew Kostrzewa edited comment on HDFS-15860 at 11/29/22 2:55 PM:
---------------------------------------------------------------------
[~elliot007] I've found a workaround/solution for my case by making my custom
authentication handler inherit from {{AltKerberosAuthenticationHandler}}
(originally I inherited from {{{}AuthenticationHandler{}}}) which effectively
enabled SPNEGO for image transfer between NNs and kept my authentication scheme
for Hadoop UIs. (details
[here|https://hadoop.apache.org/docs/stable/hadoop-auth/Configuration.html] in
section "AltKerberos Configuration")
[~prabhujoseph] [~eyang] I found this was changed in
https://issues.apache.org/jira/browse/HADOOP-16314 and though I understand the
rationale I could not find a clear information what should developers of custom
authentication handlers do in order to not hit this problem. Do you have any
recommendations?
was (Author: kostrzewa):
[~elliot007] I've found a workaround/solution for my case by making my custom
authentication handler inherit from {{AltKerberosAuthenticationHandler}} which
effectively enabled SPNEGO for image transfer between NNs and kept my
authentication scheme for Hadoop UIs. (details
[here|https://hadoop.apache.org/docs/stable/hadoop-auth/Configuration.html] in
section "AltKerberos Configuration")
[~prabhujoseph] [~eyang] I found this was changed in
https://issues.apache.org/jira/browse/HADOOP-16314 and though I understand the
rationale I could not find a clear information what should developers of custom
authentication handlers do in order to not hit this problem. Do you have any
recommendations?
> Standby Namenode bootstrap fails due to custom authentication handler being
> run for /imagetransfer endpoint
> -----------------------------------------------------------------------------------------------------------
>
> Key: HDFS-15860
> URL: https://issues.apache.org/jira/browse/HDFS-15860
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha
> Affects Versions: 3.3.0
> Environment: CentOS Linux release 7.7.1908 (Core)
> Reporter: Zbigniew Kostrzewa
> Priority: Major
>
> I have extended Hadoop (YARN, HDFS, ...) web UIs with basic authentication by
> implementing a custom authentication handler class and configuring it via
> `hadoop.http.authentication.type` in core-site.xml.
> Everything was working fine with Hadoop 2.x and 3.2.2. However, when I tried
> switching to 3.3.0 standby Namenode failed during bootstrap when trying to
> get fsimage:
> {noformat}
> 2021-02-25 21:18:52,356 ERROR [main]
> org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
> java.io.IOException: java.lang.RuntimeException:
> org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer
> servlet at http://XXXXX/imagetransfer?ge
> timage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
> failed with status code 401
> Response message:
> Authentication required
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:549)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1728)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer
> servlet at http://XXXXX/imagetransfer?getimage=1&t
> xid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
> failed with status code 401
> Response message:
> Authentication required
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:127)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:121)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:121)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:544)
> ... 2 more
> Caused by: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image
> transfer servlet at
> http://XXXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209
> 174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
> failed with status code 401
> Response message:
> Authentication required
> at org.apache.hadoop.hdfs.server.common.Util.doGetUrl(Util.java:168)
> at
> org.apache.hadoop.hdfs.server.namenode.TransferFsImage.doGetUrl(TransferFsImage.java:441)
> at
> org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:436)
> at
> org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:123)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.downloadImage(BootstrapStandby.java:357)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.doRun(BootstrapStandby.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.access$000(BootstrapStandby.java:82)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:125)
> ... 8 more
> {noformat}
> It seems that in 3.3.0 my custom handler was being invoked for
> `/imagetransfer` endpoint, from active NN:
> {noformat}
> 2021-02-25 21:18:52,328 DEBUG [qtp2033968586-22]
> org.apache.hadoop.security.authentication.server.AuthenticationFilter:
> Request
> [http://XXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true]
> triggering authentication. handler: class
> hadoop_auth_handler.BasicAuthenticationHandler
> {noformat}
> which was not the case for 3.2.2 and earlier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
