Zbigniew Kostrzewa created HDFS-15860:
-----------------------------------------
Summary: Namenode bootstrap fails due to custom authentication
handler being run for /imagetransfer endpoint
Key: HDFS-15860
URL: https://issues.apache.org/jira/browse/HDFS-15860
Project: Hadoop HDFS
Issue Type: Bug
Components: ha
Affects Versions: 3.3.0
Environment: CentOS Linux release 7.7.1908 (Core)
Reporter: Zbigniew Kostrzewa
I have extended Hadoop (YARN, HDFS, ...) web UIs with basic authentication by
implementing a custom authentication handler class and configuring it via
`hadoop.http.authentication.type` in core-site.xml.
Everything was working fine with Hadoop 2.x and 3.2.2. However, when I tried
switching to 3.3.0 standby Namenode failed during bootstrap when trying to get
fsimage:
{noformat}
2021-02-25 21:18:52,356 ERROR [main]
org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.io.IOException: java.lang.RuntimeException:
org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer
servlet at http://XXXXX/imagetransfer?ge
timage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
failed with status code 401
Response message:
Authentication required
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:549)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1728)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer
servlet at http://XXXXX/imagetransfer?getimage=1&t
xid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
failed with status code 401
Response message:
Authentication required
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:127)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:121)
at
org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:121)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:544)
... 2 more
Caused by: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image
transfer servlet at
http://XXXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209
174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true
failed with status code 401
Response message:
Authentication required
at org.apache.hadoop.hdfs.server.common.Util.doGetUrl(Util.java:168)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.doGetUrl(TransferFsImage.java:441)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:436)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:123)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.downloadImage(BootstrapStandby.java:357)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.doRun(BootstrapStandby.java:239)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.access$000(BootstrapStandby.java:82)
at
org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:125)
... 8 more
{noformat}
It seems that in 3.3.0 my custom handler was being invoked for `/imagetransfer`
endpoint, from active NN:
{noformat}
2021-02-25 21:18:52,328 DEBUG [qtp2033968586-22]
org.apache.hadoop.security.authentication.server.AuthenticationFilter: Request
[http://XXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true]
triggering authentication. handler: class
hadoop_auth_handler.BasicAuthenticationHandler
{noformat}
which was not the case for 3.2.2 and earlier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
