I'm looking for any additional steps that I could take to troubleshoot the
problem below.
I have two ListHDFS processors, both using Kerberos, each pointing to a
different HDFS system.
My first process works perfectly, but my second processor is throwing the
following error:
15:53:41 UTC ERROR aef493a8-5790-4b21-8155-16d2152ab334
ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform listing
of HDFS due to org.apache.hadoop.security.AccessControlException: SIMPLE
authentication is not enabled. Available:[TOKEN, KERBEROS]:
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]
The stack crawl in the log file doesn't provide much more info:
5:53:41 UTC ERROR aef493a8-5790-4b21-8155-16d2152ab334
ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform listing
of HDFS due to org.apache.hadoop.security.AccessControlException: SIMPLE
authentication is not enabled. Available:[TOKEN, KERBEROS]:
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]
2016-06-08 15:53:41,599 ERROR [Timer-Driven Process Thread-2]
o.apache.nifi.processors.hadoop.ListHDFS
ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform listing
of HDFS due to org.apache.hadoop.security.AccessControlException: SIMPLE
authentication is not enabled. Available:[TOKEN, KERBEROS]:
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]
2016-06-08 15:53:41,600 INFO [NiFi Web Server-6415]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] to run
2016-06-08 15:53:41,604 ERROR [Timer-Driven Process Thread-2]
o.apache.nifi.processors.hadoop.ListHDFS
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]
at sun.reflect.GeneratedConstructorAccessor328.newInstance(Unknown
Source) ~[na:na]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_74]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
~[na:1.8.0_74]
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1960)
~[hadoop-hdfs-2.6.2.jar:na]
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1941)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:693)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751)
~[hadoop-hdfs-2.6.2.jar:na]
at
org.apache.nifi.processors.hadoop.ListHDFS.getStatuses(ListHDFS.java:342)
~[nifi-hdfs-processors-0.5.1.jar:0.5.1]
at
org.apache.nifi.processors.hadoop.ListHDFS.onTrigger(ListHDFS.java:270)
~[nifi-hdfs-processors-0.5.1.jar:0.5.1]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
[nifi-api-0.5.1.jar:0.5.1]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
[nifi-framework-core-0.5.1.jar:0.5.1]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
[nifi-framework-core-0.5.1.jar:0.5.1]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
[nifi-framework-core-0.5.1.jar:0.5.1]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
[nifi-framework-core-0.5.1.jar:0.5.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_74]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_74]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_74]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_74]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_74]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_74]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
Caused by: org.apache.hadoop.ipc.RemoteException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client.call(Client.java:1469)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.ipc.Client.call(Client.java:1400)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
~[hadoop-common-2.6.2.jar:na]
at com.sun.proxy.$Proxy97.getListing(Unknown Source) ~[na:na]
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:554)
~[hadoop-hdfs-2.6.2.jar:na]
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
~[na:na]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_74]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_74]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
~[hadoop-common-2.6.2.jar:na]
at com.sun.proxy.$Proxy98.getListing(Unknown Source) ~[na:na]
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1958)
~[hadoop-hdfs-2.6.2.jar:na]
... 21 common frames omitted
It's unclear to me why I would be seeing that message if I have Kerberos
configured in the processor.
Thanks All!
