Hello,
I' m trying to connect Nifi (version 1.9.2) to Impala (3.2.0-cdh6.2.0) with
DBCPConnectionPool using jdbc driver ImpalaJDBC41.jar, using Kerberos
authentication.
I Configured a controller KeytabCredentials pointing to keytab and in
nifi.properties i set nifi.kerberos.krb5.file to krb5.ini. I made a test with a
standalone java program using this krb5.ini and keytab as System properties,
and worked well.
Apparently there is a mismatch of class of logfj from Nifi and impala driver :
"Detected both log4j-over-slf4j.jar AND bound slf4j-log4j12.jar on the class
path"
Any ideas ?
Carlos
Full Exception: 2019-06-07 15:12:26,897 ERROR [Timer-Driven Process Thread-30]
o.a.nifi.processors.standard.ExecuteSQL
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2]
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2] failed to process session
due to java.lang.ExceptionInInitializerError; Processor Administratively
Yielded for 1 sec: java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError: null
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TIOStreamTransport.<clinit>(TIOStreamTransport.java:38)
at
com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown
Source)
at
com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown
Source)
at
com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown
Source)
at
com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown
Source)
at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown
Source)
at
com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown
Source)
at
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:291)
at
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2395)
at
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2381)
at
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
at
org.apache.nifi.dbcp.DBCPConnectionPool.lambda$getConnection$2(DBCPConnectionPool.java:467)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.nifi.security.krb.AbstractKerberosUser.doAs(AbstractKerberosUser.java:143)
at
org.apache.nifi.security.krb.KerberosAction.execute(KerberosAction.java:68)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:468)
at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.getConnection(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:222)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar
AND bound slf4j-log4j12.jar on the class path, preempting StackOverflowError.
See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:54)
... 46 common frames omitted
2019-06-07 15:12:26,897 WARN [Timer-Driven Process Thread-30]
o.a.n.controller.tasks.ConnectableTask Administratively Yielding
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2] due to uncaught Exception:
java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError: null
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at
com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TIOStreamTransport.<clinit>(TIOStreamTransport.java:38)
at
com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown
Source)
at
com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown
Source)
at
com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown
Source)
at
com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown
Source)
at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown
Source)
at
com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown
Source)
at
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:291)
at
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2395)
at
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2381)
at
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
at
org.apache.nifi.dbcp.DBCPConnectionPool.lambda$getConnection$2(DBCPConnectionPool.java:467)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.nifi.security.krb.AbstractKerberosUser.doAs(AbstractKerberosUser.java:143)
at
org.apache.nifi.security.krb.KerberosAction.execute(KerberosAction.java:68)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:468)
at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.getConnection(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:222)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar
AND bound slf4j-log4j12.jar on the class path, preempting StackOverflowError.
See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
at
com.cloudera.impala.jdbc41.internal.slf4j.impl.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:54)
... 46 common frames omitted
2019-06-07 15:12:27,907 ERROR [Timer-Driven Process Thread-185]
o.a.nifi.processors.standard.ExecuteSQL
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2]
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2] failed to process session
due to java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TSocket; Processor
Administratively Yielded for 1 sec: java.lang.NoClassDefFoundError: Could not
initialize class
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TSocket
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TSocket
at
com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown
Source)
at
com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown
Source)
at
com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown
Source)
at
com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown
Source)
at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown
Source)
at
com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown
Source)
at
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:291)
at
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2395)
at
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2381)
at
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
at
org.apache.nifi.dbcp.DBCPConnectionPool.lambda$getConnection$2(DBCPConnectionPool.java:467)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.nifi.security.krb.AbstractKerberosUser.doAs(AbstractKerberosUser.java:143)
at
org.apache.nifi.security.krb.KerberosAction.execute(KerberosAction.java:68)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:468)
at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.getConnection(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:222)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2019-06-07 15:12:27,908 WARN [Timer-Driven Process Thread-185]
o.a.n.controller.tasks.ConnectableTask Administratively Yielding
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2] due to uncaught Exception:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TSocket
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TSocket
at
com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown
Source)
at
com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown
Source)
at
com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown
Source)
at
com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown
Source)
at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown
Source)
at
com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown
Source)
at
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:291)
at
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2395)
at
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2381)
at
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
at
org.apache.nifi.dbcp.DBCPConnectionPool.lambda$getConnection$2(DBCPConnectionPool.java:467)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.nifi.security.krb.AbstractKerberosUser.doAs(AbstractKerberosUser.java:143)
at
org.apache.nifi.security.krb.KerberosAction.execute(KerberosAction.java:68)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:468)
at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.getConnection(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:222)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2019-06-07 15:12:28,192 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@7457006c // Another save pending =
false
2019-06-07 15:12:28,525 INFO [NiFi Web Server-494]
o.a.n.c.s.StandardProcessScheduler Stopping
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2]
2019-06-07 15:12:28,526 INFO [NiFi Web Server-494]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.ExecuteSQL
2019-06-07 15:12:28,526 INFO [Timer-Driven Process Thread-150]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
ExecuteSQL[id=10231224-1c5a-1337-ab8d-df3308c759c2] to run
