Hi dev Team
Spark-25355 added the functionality of the proxy user on K8s . However
proxy user on K8s with Kerberized HDFS is not working . It is throwing
exception and
22/04/21 17:50:30 WARN Client: Exception encountered while connecting to
the server : org.apache.hadoop.security.AccessControlException: Client
cannot authenticate via:[TOKEN, KERBEROS]
Exception in thread "main" java.net.ConnectException: Call From <driverpod>
to <namenode> failed on connection exception: java.net.ConnectException:
Connection refused; For more details see: http://
wiki.apache.org/hadoop/ConnectionRefused
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:831)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:755)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1501)
at org.apache.hadoop.ipc.Client.call(Client.java:1443)
at org.apache.hadoop.ipc.Client.call(Client.java:1353)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy14.getFileInfo(Unknown Source)
at
On debugging deep , we found the proxy user doesn't have access to
delegation tokens in case of K8s .SparkSubmit.submit explicitly creating
the proxy user and this user doesn't have delegation token.
Please help me with the same.
Regards
Pralabh Kumar
