nifi git commit: NIFI-5354 Changing ranger.version to 1.0.0 and ranger.hadoop.version to 3.0.0
Repository: nifi
Updated Branches:
refs/heads/master a14dbc7db -> 3fd45ac4c
NIFI-5354 Changing ranger.version to 1.0.0 and ranger.hadoop.version to 3.0.0
Signed-off-by: Yolanda Davis
This closes #2827
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/3fd45ac4
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/3fd45ac4
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/3fd45ac4
Branch: refs/heads/master
Commit: 3fd45ac4c4155311966f675c6a504d683796b172
Parents: a14dbc7
Author: Bryan Bende
Authored: Fri Jun 29 10:33:10 2018 -0400
Committer: Yolanda Davis
Committed: Fri Jun 29 23:36:54 2018 -0400
--
.../nifi-ranger-bundle/nifi-ranger-plugin/pom.xml | 17 +++--
pom.xml| 2 +-
2 files changed, 16 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/3fd45ac4/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/pom.xml
--
diff --git a/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/pom.xml
b/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/pom.xml
index 05ba7ed..384997e 100644
--- a/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/pom.xml
+++ b/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/pom.xml
@@ -25,7 +25,7 @@
nifi-ranger-plugin
jar
-2.7.3
+3.0.0
@@ -91,10 +91,23 @@
findbugs-annotations
1.3.9-1
+
org.apache.hadoop
hadoop-client
-${hadoop.version}
+${ranger.hadoop.version}
+
+
+
+org.apache.hadoop
+hadoop-common
+${ranger.hadoop.version}
+
+
+org.apache.hadoop
+hadoop-auth
+${ranger.hadoop.version}
org.apache.nifi
http://git-wip-us.apache.org/repos/asf/nifi/blob/3fd45ac4/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 74431cd..b1f5250 100644
--- a/pom.xml
+++ b/pom.xml
@@ -92,7 +92,7 @@
UTF-8
2014
1.7.25
-0.7.1
+1.0.0
9.4.3.v20170317
nifi-registry git commit: NIFIREG-180 Added db configuration properties to the docker image.
Repository: nifi-registry
Updated Branches:
refs/heads/master 6abbfe1c6 -> 59348b5aa
NIFIREG-180 Added db configuration properties to the docker image.
This closes #128.
Signed-off-by: Aldrin Piri
Project: http://git-wip-us.apache.org/repos/asf/nifi-registry/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-registry/commit/59348b5a
Tree: http://git-wip-us.apache.org/repos/asf/nifi-registry/tree/59348b5a
Diff: http://git-wip-us.apache.org/repos/asf/nifi-registry/diff/59348b5a
Branch: refs/heads/master
Commit: 59348b5aa7c69c1d35e6b50e56183372b1b86220
Parents: 6abbfe1
Author: Mike
Authored: Fri Jun 29 09:51:10 2018 -0400
Committer: Aldrin Piri
Committed: Fri Jun 29 12:39:05 2018 -0400
--
nifi-registry-docker/dockerhub/README.md| 12 ++
nifi-registry-docker/dockerhub/sh/start.sh | 2 ++
.../dockerhub/sh/update_database.sh | 24
3 files changed, 38 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/nifi-registry/blob/59348b5a/nifi-registry-docker/dockerhub/README.md
--
diff --git a/nifi-registry-docker/dockerhub/README.md
b/nifi-registry-docker/dockerhub/README.md
index e4f7823..860c6dc 100644
--- a/nifi-registry-docker/dockerhub/README.md
+++ b/nifi-registry-docker/dockerhub/README.md
@@ -123,3 +123,15 @@ volume to provide certificates on the host system to the
container instance.
-e LDAP_TLS_TRUSTSTORE: ''
-e LDAP_TLS_TRUSTSTORE_PASSWORD: ''
-e LDAP_TLS_TRUSTSTORE_TYPE: ''
+
+### The following, optional environment variables can be used to configure the
database
+
+| nifi-registry.properties entry | Variable |
+|||
+| nifi.registry.db.url | NIFI_REGISTRY_DB_URL |
+| nifi.registry.db.driver.class | NIFI_REGISTRY_DB_CLASS |
+| nifi.registry.db.driver.directory | NIFI_REGISTRY_DB_DIR |
+| nifi.registry.db.driver.username | NIFI_REGISTRY_DB_USER |
+| nifi.registry.db.driver.password | NIFI_REGISTRY_DB_PASS |
+| nifi.registry.db.driver.maxConnections | NIFI_REGISTRY_DB_MAX_CONNS |
+| nifi.registry.db.sql.debug | NIFI_REGISTRY_DB_DEBUG_SQL |
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/nifi-registry/blob/59348b5a/nifi-registry-docker/dockerhub/sh/start.sh
--
diff --git a/nifi-registry-docker/dockerhub/sh/start.sh
b/nifi-registry-docker/dockerhub/sh/start.sh
index 618d229..0d725ad 100755
--- a/nifi-registry-docker/dockerhub/sh/start.sh
+++ b/nifi-registry-docker/dockerhub/sh/start.sh
@@ -23,6 +23,8 @@ scripts_dir='/opt/nifi-registry/scripts'
prop_replace 'nifi.registry.web.http.port'
"${NIFI_REGISTRY_WEB_HTTP_PORT:-18080}"
prop_replace 'nifi.registry.web.http.host'
"${NIFI_REGISTRY_WEB_HTTP_HOST:-$HOSTNAME}"
+. ${scripts_dir}/update_database.sh
+
# Check if we are secured or unsecured
case ${AUTH} in
tls)
http://git-wip-us.apache.org/repos/asf/nifi-registry/blob/59348b5a/nifi-registry-docker/dockerhub/sh/update_database.sh
--
diff --git a/nifi-registry-docker/dockerhub/sh/update_database.sh
b/nifi-registry-docker/dockerhub/sh/update_database.sh
new file mode 100644
index 000..c1c3c6f
--- /dev/null
+++ b/nifi-registry-docker/dockerhub/sh/update_database.sh
@@ -0,0 +1,24 @@
+#!/bin/sh -e
+
+#Licensed to the Apache Software Foundation (ASF) under one or more
+#contributor license agreements. See the NOTICE file distributed with
+#this work for additional information regarding copyright ownership.
+#The ASF licenses this file to You under the Apache License, Version 2.0
+#(the "License"); you may not use this file except in compliance with
+#the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+
+prop_replace 'nifi.registry.db.url'
"${NIFI_REGISTRY_DB_URL:-jdbc:h2:./database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE}"
+prop_replace 'nifi.registry.db.driver.class'
"${NIFI_REGISTRY_DB_CLASS:-org.h2.Driver}"
+prop_replace 'nifi.registry.db.driver.directory'
"${NIFI_REGISTRY_DB_DIR:-}"
+prop_replace
[nifi-minifi] Git Push Summary
Repository: nifi-minifi Updated Tags: refs/tags/minifi-0.5.0-RC2 05f516d3d -> 58b8c598c
[2/2] nifi-minifi git commit: MINIFI-458-RC2prepare for next development iteration
MINIFI-458-RC2prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/16af0f48 Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/16af0f48 Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/16af0f48 Branch: refs/heads/MINIFI-458-RC2 Commit: 16af0f48bd7fc1d8fe445776eb2f50a64f1d96c0 Parents: 58b8c59 Author: Jeremy Dyer Authored: Thu Jun 28 12:30:50 2018 -0400 Committer: Jeremy Dyer Committed: Thu Jun 28 12:30:50 2018 -0400 -- minifi-api/pom.xml | 2 +- minifi-assembly/pom.xml | 4 +- minifi-bootstrap/pom.xml| 2 +- minifi-c2/minifi-c2-api/pom.xml | 2 +- minifi-c2/minifi-c2-assembly/pom.xml| 2 +- .../minifi-c2-cache-filesystem/pom.xml | 2 +- .../minifi-c2-cache/minifi-c2-cache-s3/pom.xml | 2 +- minifi-c2/minifi-c2-cache/pom.xml | 2 +- minifi-c2/minifi-c2-docker/pom.xml | 4 +- minifi-c2/minifi-c2-integration-tests/pom.xml | 2 +- minifi-c2/minifi-c2-jetty/pom.xml | 2 +- .../minifi-c2-provider-cache/pom.xml| 2 +- .../minifi-c2-provider-delegating/pom.xml | 2 +- .../minifi-c2-provider-nifi-rest/pom.xml| 2 +- .../minifi-c2-provider-util/pom.xml | 2 +- minifi-c2/minifi-c2-provider/pom.xml| 2 +- minifi-c2/minifi-c2-service/pom.xml | 2 +- minifi-c2/pom.xml | 2 +- minifi-commons/minifi-commons-schema/pom.xml| 2 +- minifi-commons/minifi-utils/pom.xml | 2 +- minifi-commons/pom.xml | 2 +- minifi-docker/pom.xml | 4 +- minifi-docs/pom.xml | 2 +- minifi-integration-tests/pom.xml| 2 +- .../minifi-framework-nar/pom.xml| 2 +- .../minifi-framework-core/pom.xml | 2 +- .../minifi-framework/minifi-nar-utils/pom.xml | 2 +- .../minifi-framework/minifi-resources/pom.xml | 2 +- .../minifi-framework/minifi-runtime/pom.xml | 2 +- .../minifi-framework/pom.xml| 2 +- .../minifi-framework-bundle/pom.xml | 2 +- .../minifi-provenance-reporting-nar/pom.xml | 2 +- .../minifi-provenance-reporting-bundle/pom.xml | 2 +- .../pom.xml | 2 +- .../minifi-provenance-repositories/pom.xml | 2 +- .../minifi-provenance-repository-nar/pom.xml| 2 +- .../minifi-provenance-repository-bundle/pom.xml | 2 +- .../minifi-ssl-context-service-nar/pom.xml | 2 +- minifi-nar-bundles/minifi-standard-nar/pom.xml | 2 +- .../minifi-update-attribute-nar/pom.xml | 2 +- minifi-nar-bundles/pom.xml | 2 +- minifi-toolkit/minifi-toolkit-assembly/pom.xml | 2 +- .../minifi-toolkit-configuration/pom.xml| 2 +- minifi-toolkit/pom.xml | 2 +- pom.xml | 42 ++-- 45 files changed, 68 insertions(+), 68 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/16af0f48/minifi-api/pom.xml -- diff --git a/minifi-api/pom.xml b/minifi-api/pom.xml index c0080ce..6dcf082 100644 --- a/minifi-api/pom.xml +++ b/minifi-api/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi org.apache.nifi.minifi -0.5.0 +0.6.0-SNAPSHOT minifi-api jar http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/16af0f48/minifi-assembly/pom.xml -- diff --git a/minifi-assembly/pom.xml b/minifi-assembly/pom.xml index 1054ec1..f44c0d4 100644 --- a/minifi-assembly/pom.xml +++ b/minifi-assembly/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi org.apache.nifi.minifi -0.5.0 +0.6.0-SNAPSHOT minifi-assembly pom @@ -185,7 +185,7 @@ limitations under the License. org.apache.nifi.minifi minifi-provenance-reporting-nar nar -0.5.0 +0.6.0-SNAPSHOT http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/16af0f48/minifi-bootstrap/pom.xml -- diff --git a/minifi-bootstrap/pom.xml b/minifi-bootstrap/pom.xml index 253d68f..baddf27 100644 --- a/minifi-bootstrap/pom.xml +++ b/minifi-bootstrap/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi org.apache.nifi.minifi -0.5.0 +0.6.0-SNAPSHOT
[1/2] nifi-minifi git commit: NIFI-458-RC2prepare release minifi-0.5.0-RC2
Repository: nifi-minifi Updated Branches: refs/heads/MINIFI-458-RC2 05f516d3d -> 16af0f48b NIFI-458-RC2prepare release minifi-0.5.0-RC2 Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/58b8c598 Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/58b8c598 Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/58b8c598 Branch: refs/heads/MINIFI-458-RC2 Commit: 58b8c598c0866c8f1200164ab14f3df0d632d522 Parents: 05f516d Author: Jeremy Dyer Authored: Thu Jun 28 12:28:30 2018 -0400 Committer: Jeremy Dyer Committed: Thu Jun 28 12:28:30 2018 -0400 -- minifi-api/pom.xml | 2 +- minifi-assembly/pom.xml | 4 +- minifi-bootstrap/pom.xml| 2 +- minifi-c2/minifi-c2-api/pom.xml | 2 +- minifi-c2/minifi-c2-assembly/pom.xml| 2 +- .../minifi-c2-cache-filesystem/pom.xml | 2 +- .../minifi-c2-cache/minifi-c2-cache-s3/pom.xml | 2 +- minifi-c2/minifi-c2-cache/pom.xml | 2 +- minifi-c2/minifi-c2-docker/pom.xml | 4 +- minifi-c2/minifi-c2-integration-tests/pom.xml | 2 +- minifi-c2/minifi-c2-jetty/pom.xml | 2 +- .../minifi-c2-provider-cache/pom.xml| 2 +- .../minifi-c2-provider-delegating/pom.xml | 2 +- .../minifi-c2-provider-nifi-rest/pom.xml| 2 +- .../minifi-c2-provider-util/pom.xml | 2 +- minifi-c2/minifi-c2-provider/pom.xml| 2 +- minifi-c2/minifi-c2-service/pom.xml | 2 +- minifi-c2/pom.xml | 2 +- minifi-commons/minifi-commons-schema/pom.xml| 2 +- minifi-commons/minifi-utils/pom.xml | 2 +- minifi-commons/pom.xml | 2 +- minifi-docker/pom.xml | 4 +- minifi-docs/pom.xml | 2 +- minifi-integration-tests/pom.xml| 2 +- .../minifi-framework-nar/pom.xml| 2 +- .../minifi-framework-core/pom.xml | 2 +- .../minifi-framework/minifi-nar-utils/pom.xml | 2 +- .../minifi-framework/minifi-resources/pom.xml | 2 +- .../minifi-framework/minifi-runtime/pom.xml | 2 +- .../minifi-framework/pom.xml| 2 +- .../minifi-framework-bundle/pom.xml | 2 +- .../minifi-provenance-reporting-nar/pom.xml | 2 +- .../minifi-provenance-reporting-bundle/pom.xml | 2 +- .../pom.xml | 2 +- .../minifi-provenance-repositories/pom.xml | 2 +- .../minifi-provenance-repository-nar/pom.xml| 2 +- .../minifi-provenance-repository-bundle/pom.xml | 2 +- .../minifi-ssl-context-service-nar/pom.xml | 2 +- minifi-nar-bundles/minifi-standard-nar/pom.xml | 2 +- .../minifi-update-attribute-nar/pom.xml | 2 +- minifi-nar-bundles/pom.xml | 2 +- minifi-toolkit/minifi-toolkit-assembly/pom.xml | 2 +- .../minifi-toolkit-configuration/pom.xml| 2 +- minifi-toolkit/pom.xml | 2 +- pom.xml | 42 ++-- 45 files changed, 68 insertions(+), 68 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/58b8c598/minifi-api/pom.xml -- diff --git a/minifi-api/pom.xml b/minifi-api/pom.xml index 54b26d3..c0080ce 100644 --- a/minifi-api/pom.xml +++ b/minifi-api/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi org.apache.nifi.minifi -0.5.0-SNAPSHOT +0.5.0 minifi-api jar http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/58b8c598/minifi-assembly/pom.xml -- diff --git a/minifi-assembly/pom.xml b/minifi-assembly/pom.xml index 3d651a1..1054ec1 100644 --- a/minifi-assembly/pom.xml +++ b/minifi-assembly/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi org.apache.nifi.minifi -0.5.0-SNAPSHOT +0.5.0 minifi-assembly pom @@ -185,7 +185,7 @@ limitations under the License. org.apache.nifi.minifi minifi-provenance-reporting-nar nar -0.5.0-SNAPSHOT +0.5.0 http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/58b8c598/minifi-bootstrap/pom.xml -- diff --git a/minifi-bootstrap/pom.xml b/minifi-bootstrap/pom.xml index 3c6f5dc..253d68f 100644 --- a/minifi-bootstrap/pom.xml +++ b/minifi-bootstrap/pom.xml @@ -20,7 +20,7 @@ limitations under the License. minifi
nifi git commit: NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Repository: nifi
Updated Branches:
refs/heads/master c1083dfb6 -> a14dbc7db
NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Signed-off-by: Pierre Villard
This closes #2824.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/a14dbc7d
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/a14dbc7d
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/a14dbc7d
Branch: refs/heads/master
Commit: a14dbc7dbe20d31e919d52e349efbabc73caf19f
Parents: c1083df
Author: patricker
Authored: Fri Jun 29 08:33:51 2018 -0600
Committer: Pierre Villard
Committed: Fri Jun 29 16:47:55 2018 +0200
--
.../java/org/apache/nifi/processors/standard/ExecuteSQL.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/a14dbc7d/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
index af22fc3..66c2338 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
@@ -92,7 +92,9 @@ import static
org.apache.nifi.processors.standard.util.JdbcCommon.USE_AVRO_LOGIC
})
@WritesAttributes({
@WritesAttribute(attribute="executesql.row.count", description = "Contains
the number of rows returned in the select query"),
-@WritesAttribute(attribute="executesql.query.duration", description =
"Duration of the query in milliseconds"),
+@WritesAttribute(attribute="executesql.query.duration", description =
"Combined duration of the query execution time and fetch time in milliseconds"),
+@WritesAttribute(attribute="executesql.query.executiontime", description =
"Duration of the query execution time in milliseconds"),
+@WritesAttribute(attribute="executesql.query.fetchtime", description =
"Duration of the result set fetch time in milliseconds"),
@WritesAttribute(attribute="executesql.resultset.index", description =
"Assuming multiple result sets are returned, "
+ "the zero based index of this result set.")
})
nifi git commit: NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Repository: nifi
Updated Branches:
refs/heads/master 6508c191b -> c1083dfb6
NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Signed-off-by: Pierre Villard
This closes #2817.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/c1083dfb
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/c1083dfb
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/c1083dfb
Branch: refs/heads/master
Commit: c1083dfb62185567dca0b3bcd6b1202cd34093dd
Parents: 6508c19
Author: patricker
Authored: Tue Jun 26 13:27:11 2018 -0600
Committer: Pierre Villard
Committed: Fri Jun 29 14:59:27 2018 +0200
--
.../nifi/processors/standard/ExecuteSQL.java | 18 ++
.../nifi/processors/standard/TestExecuteSQL.java | 8
2 files changed, 22 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/c1083dfb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
index ac93feb..af22fc3 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
@@ -100,6 +100,8 @@ public class ExecuteSQL extends AbstractProcessor {
public static final String RESULT_ROW_COUNT = "executesql.row.count";
public static final String RESULT_QUERY_DURATION =
"executesql.query.duration";
+public static final String RESULT_QUERY_EXECUTION_TIME =
"executesql.query.executiontime";
+public static final String RESULT_QUERY_FETCH_TIME =
"executesql.query.fetchtime";
public static final String RESULTSET_INDEX = "executesql.resultset.index";
// Relationships
@@ -203,7 +205,6 @@ public class ExecuteSQL extends AbstractProcessor {
final Boolean useAvroLogicalTypes =
context.getProperty(USE_AVRO_LOGICAL_TYPES).asBoolean();
final Integer defaultPrecision =
context.getProperty(DEFAULT_PRECISION).evaluateAttributeExpressions(fileToProcess).asInteger();
final Integer defaultScale =
context.getProperty(DEFAULT_SCALE).evaluateAttributeExpressions(fileToProcess).asInteger();
-final StopWatch stopWatch = new StopWatch(true);
final String selectQuery;
if (context.getProperty(SQL_SELECT_QUERY).isSet()) {
selectQuery =
context.getProperty(SQL_SELECT_QUERY).evaluateAttributeExpressions(fileToProcess).getValue();
@@ -224,7 +225,12 @@ public class ExecuteSQL extends AbstractProcessor {
JdbcCommon.setParameters(st, fileToProcess.getAttributes());
}
logger.debug("Executing query {}", new Object[]{selectQuery});
+final StopWatch executionTime = new StopWatch(true);
+
boolean hasResults = st.execute();
+
+long executionTimeElapsed =
executionTime.getElapsed(TimeUnit.MILLISECONDS);
+
boolean hasUpdateCount = st.getUpdateCount() != -1;
while(hasResults || hasUpdateCount) {
@@ -238,6 +244,8 @@ public class ExecuteSQL extends AbstractProcessor {
resultSetFF = session.putAllAttributes(resultSetFF,
fileToProcess.getAttributes());
}
+final StopWatch fetchTime = new StopWatch(true);
+
final AtomicLong nrOfRows = new AtomicLong(0L);
resultSetFF = session.write(resultSetFF, out -> {
try {
@@ -255,17 +263,19 @@ public class ExecuteSQL extends AbstractProcessor {
}
});
-long duration =
stopWatch.getElapsed(TimeUnit.MILLISECONDS);
+long fetchTimeElapsed =
fetchTime.getElapsed(TimeUnit.MILLISECONDS);
// set attribute how many rows were selected
resultSetFF = session.putAttribute(resultSetFF,
RESULT_ROW_COUNT, String.valueOf(nrOfRows.get()));
-resultSetFF = session.putAttribute(resultSetFF,
RESULT_QUERY_DURATION, String.valueOf(duration));
+resultSetFF = session.putAttribute(resultSetFF,
RESULT_QUERY_DURATION, String.valueOf(executionTimeElapsed + fetchTimeElapsed));
+resultSetFF = session.putAttribute(resultSetFF,
RESULT_QUERY_EXECUTION_TIME,
nifi git commit: NIFI-5343: Fix GetMongo to properly reads charset from FlowFile attributes
Repository: nifi
Updated Branches:
refs/heads/master e741ba1ce -> 6508c191b
NIFI-5343: Fix GetMongo to properly reads charset from FlowFile attributes
Signed-off-by: Pierre Villard
This closes #2818.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/6508c191
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/6508c191
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/6508c191
Branch: refs/heads/master
Commit: 6508c191bffc855bf9c8d11de06ef1eca3d83d26
Parents: e741ba1
Author: zenfenan
Authored: Wed Jun 27 15:30:11 2018 +0530
Committer: Pierre Villard
Committed: Fri Jun 29 14:56:24 2018 +0200
--
.../apache/nifi/processors/mongodb/GetMongo.java | 4 ++--
.../nifi/processors/mongodb/GetMongoIT.java | 18 ++
2 files changed, 20 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/6508c191/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
--
diff --git
a/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
b/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
index 356f3f4..e744577 100644
---
a/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
+++
b/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
@@ -281,7 +281,7 @@ public class GetMongo extends AbstractMongoProcessor {
log.debug("Writing batch...");
}
String payload = buildBatch(batch,
jsonTypeSetting, usePrettyPrint);
-writeBatch(payload, null, context, session,
attributes, REL_SUCCESS);
+writeBatch(payload, input, context, session,
attributes, REL_SUCCESS);
batch = new ArrayList<>();
} catch (Exception ex) {
getLogger().error("Error building batch", ex);
@@ -290,7 +290,7 @@ public class GetMongo extends AbstractMongoProcessor {
}
if (batch.size() > 0) {
try {
-writeBatch(buildBatch(batch, jsonTypeSetting,
usePrettyPrint), null, context, session, attributes, REL_SUCCESS);
+writeBatch(buildBatch(batch, jsonTypeSetting,
usePrettyPrint), input, context, session, attributes, REL_SUCCESS);
} catch (Exception ex) {
getLogger().error("Error sending remainder of
batch", ex);
}
http://git-wip-us.apache.org/repos/asf/nifi/blob/6508c191/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoIT.java
--
diff --git
a/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoIT.java
b/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoIT.java
index 4bb0078..2c9b6d2 100644
---
a/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoIT.java
+++
b/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoIT.java
@@ -427,6 +427,24 @@ public class GetMongoIT {
runner.assertTransferCount(GetMongo.REL_ORIGINAL, 0);
runner.assertTransferCount(GetMongo.REL_SUCCESS, 3);
}
+
+@Test
+public void testReadCharsetWithEL() {
+String query = "{ \"c\": { \"$gte\": 4 }}";
+Map attrs = new HashMap<>();
+attrs.put("charset", "UTF-8");
+
+runner.setProperty(GetMongo.CHARSET, "${charset}");
+runner.setProperty(GetMongo.BATCH_SIZE, "2");
+runner.setProperty(GetMongo.RESULTS_PER_FLOWFILE, "2");
+
+runner.setIncomingConnection(true);
+runner.enqueue(query, attrs);
+runner.run(1, true, true);
+runner.assertTransferCount(GetMongo.REL_FAILURE, 0);
+runner.assertTransferCount(GetMongo.REL_ORIGINAL, 1);
+runner.assertTransferCount(GetMongo.REL_SUCCESS, 1);
+}
/*
* End query read behavior tests
*/
nifi git commit: NIFI-5349: Fixed EL handling in Initial Max Value props for DB Fetch processors
Repository: nifi
Updated Branches:
refs/heads/master 3be08b7e4 -> e741ba1ce
NIFI-5349: Fixed EL handling in Initial Max Value props for DB Fetch processors
Signed-off-by: Pierre Villard
This closes #2822.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/e741ba1c
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/e741ba1c
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/e741ba1c
Branch: refs/heads/master
Commit: e741ba1ce4bf1ec8ecdf9aea1e0040deda430775
Parents: 3be08b7
Author: Matthew Burgess
Authored: Thu Jun 28 15:02:58 2018 -0400
Committer: Pierre Villard
Committed: Fri Jun 29 14:51:59 2018 +0200
--
.../AbstractDatabaseFetchProcessor.java | 30 +---
.../processors/standard/GenerateTableFetch.java | 24 ++-
.../processors/standard/QueryDatabaseTable.java | 20 ++-
.../standard/QueryDatabaseTableTest.java| 56 +++
.../standard/TestGenerateTableFetch.java| 161 +++
5 files changed, 261 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/e741ba1c/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
index 924c7da..e13f9de 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
@@ -21,7 +21,6 @@ import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.components.ValidationContext;
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.dbcp.DBCPService;
-import org.apache.nifi.expression.AttributeExpression;
import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
@@ -220,18 +219,6 @@ public abstract class AbstractDatabaseFetchProcessor
extends AbstractSessionFact
.build();
}
-@Override
-protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final
String propertyDescriptorName) {
-return new PropertyDescriptor.Builder()
-.name(propertyDescriptorName)
-.required(false)
-
.addValidator(StandardValidators.createAttributeExpressionLanguageValidator(AttributeExpression.ResultType.STRING,
true))
-
.addValidator(StandardValidators.ATTRIBUTE_KEY_PROPERTY_NAME_VALIDATOR)
-
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
-.dynamic(true)
-.build();
-}
-
// A common validation procedure for DB fetch processors, it stores
whether the Table Name and/or Max Value Column properties have expression
language
protected Collection customValidate(ValidationContext
validationContext) {
// For backwards-compatibility, keep track of whether the table name
and max-value column properties are dynamic (i.e. has expression language)
@@ -540,19 +527,16 @@ public abstract class AbstractDatabaseFetchProcessor
extends AbstractSessionFact
return sb.toString();
}
-protected Map getDefaultMaxValueProperties(final
Map properties){
-final Map defaultMaxValues = new HashMap<>();
+protected Map getDefaultMaxValueProperties(final
ProcessContext context, final FlowFile flowFile) {
+final Map defaultMaxValues = new HashMap<>();
-for (final Map.Entry entry :
properties.entrySet()) {
-final String key = entry.getKey().getName();
+context.getProperties().forEach((k, v) -> {
+final String key = k.getName();
-if(!key.startsWith(INITIAL_MAX_VALUE_PROP_START)) {
-continue;
+if (key.startsWith(INITIAL_MAX_VALUE_PROP_START)) {
+
defaultMaxValues.put(key.substring(INITIAL_MAX_VALUE_PROP_START.length()),
context.getProperty(k).evaluateAttributeExpressions(flowFile).getValue());
}
-
-
defaultMaxValues.put(key.substring(INITIAL_MAX_VALUE_PROP_START.length()),
entry.getValue());
-}
-
+});
return defaultMaxValues;
}
}
nifi-minifi-cpp git commit: MINIFICPP-538 - only print error on actual error during shutdown, update semaphore name
Repository: nifi-minifi-cpp
Updated Branches:
refs/heads/master 4160fb2f9 -> 10eb9757b
MINIFICPP-538 - only print error on actual error during shutdown, update
semaphore name
This closes #364.
Signed-off-by: Marc Parisi
Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/commit/10eb9757
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/tree/10eb9757
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/diff/10eb9757
Branch: refs/heads/master
Commit: 10eb9757b9302646f2020d57668002286fa27887
Parents: 4160fb2
Author: Dustin Rodrigues
Authored: Sun Jun 24 13:30:02 2018 -0400
Committer: Marc Parisi
Committed: Fri Jun 29 06:44:23 2018 -0400
--
main/MiNiFiMain.cpp | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/10eb9757/main/MiNiFiMain.cpp
--
diff --git a/main/MiNiFiMain.cpp b/main/MiNiFiMain.cpp
index 65d79ed..58c71bc 100644
--- a/main/MiNiFiMain.cpp
+++ b/main/MiNiFiMain.cpp
@@ -71,7 +71,7 @@ int main(int argc, char **argv) {
std::string nifi_configuration_class_name = "yamlconfiguration";
std::string content_repo_class = "filesystemrepository";
- running = sem_open("MiNiFiMain", O_CREAT, 0644, 0);
+ running = sem_open("/MiNiFiMain", O_CREAT, 0644, 0);
if (running == SEM_FAILED || running == 0) {
logger->log_error("could not initialize semaphore");
@@ -213,10 +213,14 @@ int main(int argc, char **argv) {
* yield without the need for a more complex construct and
* a spin lock
*/
- if (sem_wait(running) != -1)
+ if (sem_wait(running) == -1)
perror("sem_wait");
- sem_unlink("MiNiFiMain");
+ if (sem_close(running) == -1)
+perror("sem_close");
+
+ if (sem_unlink("/MiNiFiMain") == -1)
+perror("sem_unlink");
/**
* Trigger unload -- wait stop_wait_time
nifi-minifi-cpp git commit: MINIFICPP-546 - exit code of 0 for minifi.sh status when process isn't running
Repository: nifi-minifi-cpp
Updated Branches:
refs/heads/master 0f5e2fa81 -> 4160fb2f9
MINIFICPP-546 - exit code of 0 for minifi.sh status when process isn't running
This closes #366.
Signed-off-by: Marc Parisi
Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/commit/4160fb2f
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/tree/4160fb2f
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/diff/4160fb2f
Branch: refs/heads/master
Commit: 4160fb2f99d10be6254c2dcad19826579ead8068
Parents: 0f5e2fa
Author: Dustin Rodrigues
Authored: Tue Jun 26 23:00:57 2018 -0400
Committer: Marc Parisi
Committed: Fri Jun 29 06:40:32 2018 -0400
--
bin/minifi.sh | 5 +
1 file changed, 1 insertion(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/4160fb2f/bin/minifi.sh
--
diff --git a/bin/minifi.sh b/bin/minifi.sh
index 8c4d1ff..1a62838 100755
--- a/bin/minifi.sh
+++ b/bin/minifi.sh
@@ -329,9 +329,6 @@ case "$1" in
${minifi_executable}
;;
status)
- # interpret status as per LSB specifications
- # see:
http://refspecs.linuxbase.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html
-
if [ "${saved_pid}" -gt 0 ]; then
if [ $(active_pid ${saved_pid}) -ne 0 ]; then
# program is dead and pid file exists
@@ -345,7 +342,7 @@ case "$1" in
else
# program is not running
echo "MiNiFi is not currently running."
-exit 3;
+exit 0;
fi
;;
update)
