LEEAHRI opened a new issue, #17485:
URL: https://github.com/apache/druid/issues/17485
Please provide a detailed title (e.g. "Broker crashes when using TopN query
with Bound filter" instead of just "Broker crashes").
### Affected Version
Druid 31.0.0
### Description
Hello,
I ran into a Kerberos authentication problem while trying to run a Hadoop
index task. The problem seems to occur during the process of getting the MR job
report. However, there is no direct issue with the task itself, and the task
completes successfully, so the problem seems to be related to the
authentication step rather than the execution of the task.
Below is the error stack trace (hadoop-client-api-3.3.6):
```
2024-11-18T15:11:03,789 WARN [task-runner-0-priority-0]
org.apache.hadoop.ipc.Client - Exception encountered while connecting to the
server {server:port}
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]
at
org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:179)
~[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:392)
~[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:561)
~[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:347)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:783)
~[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:779)
~[hadoop-client-api-3.3.6.jar:?]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_242]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_242]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
~[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:779)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:347)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1632)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client.call(Client.java:1457)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.ipc.Client.call(Client.java:1410)
[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:258)
[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:139)
[hadoop-client-api-3.3.6.jar:?]
at com.sun.proxy.$Proxy263.getJobReport(Unknown Source) [?:?]
at
org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:135)
[hadoop-client-api-3.3.6.jar:?]
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_242]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:326)
[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:430)
[hadoop-client-api-3.3.6.jar:?]
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:872)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:333)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:330)
[hadoop-client-api-3.3.6.jar:?]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_242]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_242]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:330)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job.isComplete(Job.java:614)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1737)
[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1699)
[hadoop-client-api-3.3.6.jar:?]
at
org.apache.druid.indexer.DetermineHashedPartitionsJob.run(DetermineHashedPartitionsJob.java:136)
[druid-indexing-hadoop-31.0.0-SNAPSHOT.jar:31.0.0-SNAPSHOT]
```
When I lowered the library version to hadoop-client:2.8.5 in the
hadoopDependencyCoordinates, I could not reproduce the authentication issue and
the task ran successfully with no errors.
```
"hadoopDependencyCoordinates": ["org.apache.hadoop:hadoop-client:2.8.5"]
```
Is there any guidance you can provide on how to resolve the issue with the
Kerberos authentication?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
