Hi Michael, Can you confirm that the ListHDFS that is showing the problem is pointing at a core-site.xml that has the value of "hadoop.security.authentication" set to "kerberos" ?
-Bryan On Wed, Jun 8, 2016 at 12:22 PM, Michael Dyer <[email protected]> wrote: > I'm looking for any additional steps that I could take to troubleshoot the > problem below. > > I have two ListHDFS processors, both using Kerberos, each pointing to a > different HDFS system. > > My first process works perfectly, but my second processor is throwing the > following error: > > 15:53:41 UTC ERROR aef493a8-5790-4b21-8155-16d2152ab334 > ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform > listing of HDFS due to org.apache.hadoop.security.AccessControlException: > SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]: > org.apache.hadoop.security.AccessControlException: SIMPLE authentication is > not enabled. Available:[TOKEN, KERBEROS] > > The stack crawl in the log file doesn't provide much more info: > > 5:53:41 UTC ERROR aef493a8-5790-4b21-8155-16d2152ab334 > ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform > listing of HDFS due to org.apache.hadoop.security.AccessControlException: > SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]: > org.apache.hadoop.security.AccessControlException: SIMPLE authentication is > not enabled. Available:[TOKEN, KERBEROS] > > 2016-06-08 15:53:41,599 ERROR [Timer-Driven Process Thread-2] > o.apache.nifi.processors.hadoop.ListHDFS > ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] Failed to perform listing > of HDFS due to org.apache.hadoop.security.AccessControlException: SIMPLE > authentication is not enabled. Available:[TOKEN, KERBEROS]: > org.apache.hadoop.security.AccessControlException: SIMPLE authentication is > not enabled. Available:[TOKEN, KERBEROS] > 2016-06-08 15:53:41,600 INFO [NiFi Web Server-6415] > o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling > ListHDFS[id=aef493a8-5790-4b21-8155-16d2152ab334] to run > 2016-06-08 15:53:41,604 ERROR [Timer-Driven Process Thread-2] > o.apache.nifi.processors.hadoop.ListHDFS > org.apache.hadoop.security.AccessControlException: SIMPLE authentication > is not enabled. Available:[TOKEN, KERBEROS] > at sun.reflect.GeneratedConstructorAccessor328.newInstance(Unknown > Source) ~[na:na] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > ~[na:1.8.0_74] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > ~[na:1.8.0_74] > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) > ~[hadoop-common-2.6.2.jar:na] > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73) > ~[hadoop-common-2.6.2.jar:na] > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1960) > ~[hadoop-hdfs-2.6.2.jar:na] > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1941) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:693) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > ~[hadoop-common-2.6.2.jar:na] > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751) > ~[hadoop-hdfs-2.6.2.jar:na] > at > org.apache.nifi.processors.hadoop.ListHDFS.getStatuses(ListHDFS.java:342) > ~[nifi-hdfs-processors-0.5.1.jar:0.5.1] > at > org.apache.nifi.processors.hadoop.ListHDFS.onTrigger(ListHDFS.java:270) > ~[nifi-hdfs-processors-0.5.1.jar:0.5.1] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > [nifi-api-0.5.1.jar:0.5.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139) > [nifi-framework-core-0.5.1.jar:0.5.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.5.1.jar:0.5.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.5.1.jar:0.5.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124) > [nifi-framework-core-0.5.1.jar:0.5.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_74] > at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_74] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_74] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_74] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_74] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_74] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74] > Caused by: org.apache.hadoop.ipc.RemoteException: SIMPLE authentication is > not enabled. Available:[TOKEN, KERBEROS] > at org.apache.hadoop.ipc.Client.call(Client.java:1469) > ~[hadoop-common-2.6.2.jar:na] > at org.apache.hadoop.ipc.Client.call(Client.java:1400) > ~[hadoop-common-2.6.2.jar:na] > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) > ~[hadoop-common-2.6.2.jar:na] > at com.sun.proxy.$Proxy97.getListing(Unknown Source) ~[na:na] > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:554) > ~[hadoop-hdfs-2.6.2.jar:na] > at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source) > ~[na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[na:1.8.0_74] > at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_74] > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > ~[hadoop-common-2.6.2.jar:na] > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > ~[hadoop-common-2.6.2.jar:na] > at com.sun.proxy.$Proxy98.getListing(Unknown Source) ~[na:na] > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1958) > ~[hadoop-hdfs-2.6.2.jar:na] > ... 21 common frames omitted > > It's unclear to me why I would be seeing that message if I have Kerberos > configured in the processor. > > Thanks All! >
