[ 
https://issues.apache.org/jira/browse/NIFI-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397373#comment-16397373
 ] 

ASF GitHub Bot commented on NIFI-2575:
--------------------------------------

Github user gardellajuanpablo commented on a diff in the pull request:

    https://github.com/apache/nifi/pull/2507#discussion_r174227238
  
    --- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java
 ---
    @@ -859,4 +859,33 @@ public static DateTimeFormatter 
getDateTimeFormatter(String pattern) {
             void processRow(ResultSet resultSet) throws IOException;
         }
     
    +    /**
    +     * Returns the table name.
    +     * 
    +     * @param meta
    +     *            - metadata
    +     * @return the table name or an empty string if the driver does not 
support
    +     *         {@link ResultSetMetaData#getTableName(int)}.
    +     */
    +    private static String getTableNameFromMeta(final ResultSetMetaData 
meta) {
    +        try {
    +            return meta.getTableName(1);
    +        } catch (SQLException e) {
    +            return "";
    --- End diff --
    
    Awesome feedback, thanks. It makes sense to me. 
    
    May I ask if we can apply these changes later in another JIRA? I'm asking 
because in current state, we cannot use it in Hive. But, as you describe above, 
maybe the patch, in some scenarios, it could cause problems.
    
    The changes I suggested are working now in our environment (actually it 
requires 
https://issues.apache.org/jira/browse/NIFI-2575?focusedCommentId=16382049&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16382049
 patch).  
    
    I will try to work on your feedback if you think the patch should not be 
used as is. 


> HiveQL Processors Fail due to invalid JDBC URI resolution when using 
> Zookeeper URI
> ----------------------------------------------------------------------------------
>
>                 Key: NIFI-2575
>                 URL: https://issues.apache.org/jira/browse/NIFI-2575
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Yolanda M. Davis
>            Priority: Major
>
> When configuring a HiveQL processor using the Zookeeper URL (e.g. 
> jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
>  it appears that the JDBC driver does not properly build the the uri in the 
> expected format.  This is because HS2 is storing JDBC parameters in ZK 
> (https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the 
> driver to be able to parse and use those values to configure the connection. 
> However it appears the driver is expecting zookeeper to simply return the 
> host:port and subsequently building an invalid URI.
> This problem has result in two variation of errors. The following was 
> experienced by [~mattyb149]
> {noformat}
> 2016-08-15 12:45:12,918 INFO [Timer-Driven Process Thread-2] 
> org.apache.hive.jdbc.Utils Resolved authority: 
> hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com
> 2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2] 
> org.apache.hive.jdbc.HiveConnection Will try to open client transport with 
> JDBC Uri: 
> jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2] 
> org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC 
> Uri: 
> jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:45:13,836 INFO [Timer-Driven Process Thread-2] 
> o.a.c.f.imps.CuratorFrameworkImpl Starting
> 2016-08-15 12:45:14,064 INFO [Timer-Driven Process Thread-2-EventThread] 
> o.a.c.f.state.ConnectionStateManager State change: CONNECTED
> 2016-08-15 12:45:14,182 INFO [Curator-Framework-0] 
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2016-08-15 12:45:14,337 ERROR [Timer-Driven Process Thread-2] 
> o.a.nifi.processors.hive.SelectHiveQL 
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] 
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due 
> to java.lang.reflect.UndeclaredThrowableException; rolling back session: 
> java.lang.reflect.UndeclaredThrowableException
> 2016-08-15 12:45:14,346 ERROR [Timer-Driven Process Thread-2] 
> o.a.nifi.processors.hive.SelectHiveQL
> java.lang.reflect.UndeclaredThrowableException: null
>               at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>  ~[na:na]
>               at 
> org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
>  ~[na:na]
>               at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown 
> Source) ~[na:na]
>               at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_65]
>               at java.lang.reflect.Method.invoke(Method.java:497) 
> ~[na:1.8.0_65]
>               at 
> org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
>               at 
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158) 
> ~[na:na]
>               at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>               at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_65]
>               at 
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65]
>               at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_65]
>               at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_65]
>               at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_65]
>               at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_65]
>               at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create 
> PoolableConnectionFactory (Could not open client transport for any of the 
> Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper)
>               at 
> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
>  ~[na:na]
> {noformat}
> The following error was experienced by [~YolandaMDavis]
> {noformat}
> 2016-08-15 19:22:27,338 INFO [Timer-Driven Process Thread-7] 
> org.apache.hive.jdbc.Utils Supplied authorities: 
> ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181
> 2016-08-15 19:22:27,340 INFO [Timer-Driven Process Thread-7] 
> o.a.c.f.imps.CuratorFrameworkImpl Starting
> 2016-08-15 19:22:27,360 INFO [Timer-Driven Process Thread-7-EventThread] 
> o.a.c.f.state.ConnectionStateManager State change: CONNECTED
> 2016-08-15 19:22:27,365 INFO [Timer-Driven Process Thread-7] 
> o.a.hive.jdbc.ZooKeeperHiveClientHelper Selected HiveServer2 instance with 
> uri: 
> hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=ydavis-hdp-nifi-test-2.openstacklocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
> 2016-08-15 19:22:27,365 INFO [Curator-Framework-0] 
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2016-08-15 19:22:27,371 INFO [Timer-Driven Process Thread-7] 
> org.apache.hive.jdbc.Utils Resolved authority: 
> hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=ydavis-hdp-nifi-test-2.openstacklocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
> 2016-08-15 19:22:27,374 INFO [Timer-Driven Process Thread-7] 
> org.apache.hive.jdbc.HiveConnection Will try to open client transport with 
> JDBC Uri: 
> jdbc:hive2://hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=ydavis-hdp-nifi-test-2.openstacklocal;hive.server2.thrift.port=10000;hive.server2.use.SSL=false/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 19:22:27,374 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.hive.PutHiveQL 
> PutHiveQL[id=74c17a11-0156-1000-5d7e-40ed6aa18cbb] 
> PutHiveQL[id=74c17a11-0156-1000-5d7e-40ed6aa18cbb] failed to process due to 
> java.lang.NullPointerException; rolling back session: 
> java.lang.NullPointerException
> 2016-08-15 19:22:27,383 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.hive.PutHiveQL 
> java.lang.NullPointerException: null
>       at org.apache.thrift.transport.TSocket.open(TSocket.java:170) ~[na:na]
>       at 
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266) 
> ~[na:na]
>       at 
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>  ~[na:na]
>       at 
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204) 
> ~[na:na]
>       at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176) 
> ~[na:na]
>       at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) ~[na:na]
>       at 
> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  ~[na:na]
>       at 
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
>  ~[na:na]
>       at 
> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
>  ~[na:na]
>       at 
> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
>  ~[na:na]
>       at 
> org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
>  ~[na:na]
>       at 
> org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
>  ~[na:na]
>       at 
> org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:264)
>  ~[na:na]
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_101]
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_101]
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_101]
>       at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
>       at 
> org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at com.sun.proxy.$Proxy120.getConnection(Unknown Source) ~[na:na]
>       at 
> org.apache.nifi.processors.hive.PutHiveQL.onTrigger(PutHiveQL.java:152) 
> ~[na:na]
>       at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_101]
>       at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_101]
>       at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_101]
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_101]
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_101]
>       at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> 2016-08-15 19:22:27,384 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.hive.PutHiveQL 
> PutHiveQL[id=74c17a11-0156-1000-5d7e-40ed6aa18cbb] 
> PutHiveQL[id=74c17a11-0156-1000-5d7e-40ed6aa18cbb] failed to process session 
> due to java.lang.NullPointerException: java.lang.NullPointerException
> {noformat}
> This problem appears to be resolved in the Apache 1.3 release which is not 
> yet available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to