Hi all, I need help finding the root cause on this issue. I occasionally get the following exception from SelectHiveQL and PutHiveQL processor. Sometimes the queries executed without any issue but next minute this exception could happen. Once it happened, by restarting only the NiFi does not resolve the issue. I have to reboot the Linux VM of the node and remove the flow.xml.gz file then restart nifi node.
I am using NIFI version 1.7.1 in cluster mode with total 8 nodes Here is the stacktrace 2019-05-14 20:37:30,937 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.hive.SelectHiveQL SelectHiveQL[id=365c2597-9483-302c-90d0-c7d2d7ce471d] Unable to execute HiveQL select query select max(version) as max_partition, min(version) as min_partition from Table_A for StandardFlowFileRecord[uuid=9253b1ab-b4a2-4547-b695-4dbf5f964ca2,claim=,offset=0,name=done,size=0] due to org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records.; routing to failure: org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records. org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records. at org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$2(SelectHiveQL.java:407) at org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$627/1348460623.process(Unknown Source) at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625) at org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:395) at org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:285) at org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$622/285174376.execute(Unknown Source) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106) at org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:285) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.sql.SQLException: java.io.IOException: java.lang.RuntimeException: native snappy library not available at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121) at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109) at org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:330) at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207) at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207) at org.apache.nifi.util.hive.HiveJdbcCommon.convertToCsvStream(HiveJdbcCommon.java:355) at org.apache.nifi.util.hive.HiveJdbcCommon.convertToCsvStream(HiveJdbcCommon.java:322) at org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$2(SelectHiveQL.java:401) ... 18 common frames omitted 2019-05-14 20:37:30,938 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.hive.SelectHiveQL SelectHiveQL[id=365c2597-9483-302c-90d0-c7d2d7ce471d] org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$622/285174376@54eb300d failed to process due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1557687270864-64540, container=default, section=28], offset=81777, length=280],offset=0,name=done,size=280] transfer relationship not specified; rolling back session: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1557687270864-64540, container=default, section=28], offset=81777, length=280],offset=0,name=done,size=280] transfer relationship not specified org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1557687270864-64540, container=default, section=28], offset=81777, length=280],offset=0,name=done,size=280] transfer relationship not specified at org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:264) at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:335) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:115) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106) at org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:285) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Thanks David
