[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507609#comment-16507609
]
ASF GitHub Bot commented on NIFI-5044:
--------------------------------------
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2695#discussion_r194283856
--- Diff:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/SelectHiveQL.java
---
@@ -443,31 +481,72 @@ private void onTrigger(final ProcessContext context,
final ProcessSession sessio
throw e;
}
- session.transfer(resultSetFlowFiles, REL_SUCCESS);
+ failure = executeConfigStatements(con, postQueries);
+ if (failure != null) {
+ hqlStatement = failure.getLeft();
+ if (resultSetFlowFiles != null) {
+ resultSetFlowFiles.forEach(ff -> session.remove(ff));
+ }
+ flowfile = (fileToProcess == null) ? session.create() :
fileToProcess;
+ fileToProcess = null;
+ throw failure.getRight();
+ }
+ session.transfer(resultSetFlowFiles, REL_SUCCESS);
+ if (fileToProcess != null) {
+ session.remove(fileToProcess);
+ }
} catch (final ProcessException | SQLException e) {
- logger.error("Issue processing SQL {} due to {}.", new
Object[]{selectQuery, e});
+ logger.error("Issue processing SQL {} due to {}.", new
Object[]{hqlStatement, e});
if (flowfile == null) {
// This can happen if any exceptions occur while setting
up the connection, statement, etc.
logger.error("Unable to execute HiveQL select query {} due
to {}. No FlowFile to route to failure",
- new Object[]{selectQuery, e});
+ new Object[]{hqlStatement, e});
context.yield();
} else {
if (context.hasIncomingConnection()) {
logger.error("Unable to execute HiveQL select query {}
for {} due to {}; routing to failure",
- new Object[]{selectQuery, flowfile, e});
+ new Object[]{hqlStatement, flowfile, e});
flowfile = session.penalize(flowfile);
} else {
logger.error("Unable to execute HiveQL select query {}
due to {}; routing to failure",
- new Object[]{selectQuery, e});
+ new Object[]{hqlStatement, e});
context.yield();
}
session.transfer(flowfile, REL_FAILURE);
}
- } finally {
- if (fileToProcess != null) {
- session.remove(fileToProcess);
+ }
+ }
+
+ /*
+ * Executes given queries using pre-defined connection.
+ * Returns null on success, or a query string if failed.
+ */
+ protected Pair<String,SQLException> executeConfigStatements(final
Connection con, final List<String> configQueries){
+ if (configQueries == null || configQueries.isEmpty()) {
+ return null;
+ }
+
+ for (String confSQL : configQueries) {
+ try(final Statement st = con.createStatement()){
+ st.executeQuery(confSQL);
--- End diff --
Shouldn't this be st.execute()? When I run with a single "set" statement I
get a SQLException that the query did not return a result set...
> SelectHiveQL accept only one statement
> --------------------------------------
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0
> Reporter: Davide Isoardi
> Assignee: Ed Berezitsky
> Priority: Critical
> Labels: features, patch, pull-request-available
> Attachments:
> 0001-NIFI-5044-SelectHiveQL-accept-only-one-statement.patch
>
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af-ffff-ffffee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
