[
https://issues.apache.org/jira/browse/NIFI-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736965#comment-15736965
]
ASF GitHub Bot commented on NIFI-3031:
--------------------------------------
GitHub user dstreev opened a pull request:
https://github.com/apache/nifi/pull/1316
NIFI-3031 Support Multi-Statement Scripts in the PutHiveQL Processor
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dstreev/nifi-1 NIFI-3031
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1316.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1316
----
commit 7a18054dad40e3c21a9b8c7dd760a8e283f12287
Author: David W. Streever <[email protected]>
Date: 2016-11-04T15:03:17Z
PutHiveQL and SelectHiveQL Processor enhancements. Added support for
multiple statements in a script. Options for delimiters, quotes, escaping,
include header and alternate header.
Add support in SelectHiveQL to get script content from the Flow File to
bring consistency with patterns used for PutHiveQL and support extra query
management.
Changed behavior of using Flowfile to match ExecuteSQL. Handle query
delimiter when embedded. Added test case for embedded delimiter
Formatting and License Header
PutHiveQL and SelectHiveQL Processor enhancements. Added support for
multiple statements in a script. Options for delimiters, quotes, escaping,
include header and alternate header.
Add support in SelectHiveQL to get script content from the Flow File to
bring consistency with patterns used for PutHiveQL and support extra query
management.
Changed behavior of using Flowfile to match ExecuteSQL. Handle query
delimiter when embedded. Added test case for embedded delimiter
Removing dead code.
commit 31efc23963428c389ca0d6ae01b64ad1e025040e
Author: David W. Streever <[email protected]>
Date: 2016-12-10T01:42:35Z
Comments to Clarify test case.
----
> Support Multi-Statement Scripts in the PutHiveQL Processor
> ----------------------------------------------------------
>
> Key: NIFI-3031
> URL: https://issues.apache.org/jira/browse/NIFI-3031
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Matt Burgess
>
> Trying to use the PutHiveQL processor to execute a HiveQL script that
> contains multiple statements.
> IE:
> USE my_database;
> FROM my_database_src.base_table
> INSERT OVERWRITE refined_table
> SELECT *;
> -- or --
> use my_database;
> create temporary table WORKING as
> select a,b,c from RAW;
> FROM RAW
> INSERT OVERWRITE refined_table
> SELECT *;
> The current implementation doesn't even like it when you have a semicolon at
> the end of the single statement.
> Either use a default delimiter like a semi-colon to mark the boundaries of a
> statement within the file or allow them to define there own.
> This enables the building of pipelines that are testable by not embedding
> HiveQL into a product; rather sourcing them from files. And the scripts can
> be complex. Each statement should run in a linear manner and be part of the
> same JDBC session to ensure things like "temporary" tables will work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
