[
https://issues.apache.org/jira/browse/NIFI-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262998#comment-15262998
]
ASF GitHub Bot commented on NIFI-981:
-------------------------------------
Github user bbende commented on a diff in the pull request:
https://github.com/apache/nifi/pull/384#discussion_r61502555
--- Diff:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/ExecuteHiveQL.java
---
@@ -152,8 +152,23 @@ public void process(final OutputStream out) throws
IOException {
logger.info("{} contains {} Avro records; transferring to
'success'",
new Object[]{fileToProcess, nrOfRows.get()});
- session.getProvenanceReporter().modifyContent(fileToProcess,
"Retrieved " + nrOfRows.get() + " rows",
- stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+
+ if (context.hasIncomingConnection()) {
+ // If the flow file came from an incoming connection,
issue a Modify Content provenance event
+
+
session.getProvenanceReporter().modifyContent(fileToProcess, "Retrieved " +
nrOfRows.get() + " rows",
+ stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+ } else {
+ // If we created a flow file from rows received from Hive,
issue a Receive provenance event
+ // Determine the database URL from the connection metadata
+ String url = "jdbc:hive2://unknown-host";
+ try {
+ url = con.getMetaData().getURL();
--- End diff --
In your testing did con.getMetaData().getUrl() return a value?
Testing on a VM I am always getting unknown-host in my provenance events.
I'm wondering should we just take the value from the Database Connection URL
property and use that as the URI here, instead of relying on the connection
object.
> Add support for Hive JDBC / ExecuteSQL
> --------------------------------------
>
> Key: NIFI-981
> URL: https://issues.apache.org/jira/browse/NIFI-981
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Joseph Witt
> Assignee: Matt Burgess
>
> In this mailing list thread from September 2015 "NIFI DBCP connection pool
> not working for hive" the main thrust of the converstation is to provide
> proper support for delivering data to hive. Hive's jdbc driver appears to
> have dependencies on Hadoop libraries. We need to be careful/thoughtful
> about how to best support this so that different versions of Hadoop distros
> can be supported (potentially in parallel on the same flow).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
