[nifi] branch main updated (f11c99f -> 1033e77)
This is an automated email from the ASF dual-hosted git repository. mattyb149 pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git. from f11c99f NIFI-8475: Fixed EL query parser when input contains multiple escaped expressions add 1033e77 NIFI-8686 Update administration guide with kernel 3 time_wait setting No new revisions were added by this update. Summary of changes: nifi-docs/src/main/asciidoc/administration-guide.adoc | 6 ++ 1 file changed, 6 insertions(+)
[nifi] branch main updated (786f171 -> f11c99f)
This is an automated email from the ASF dual-hosted git repository. mattyb149 pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git. from 786f171 NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail add f11c99f NIFI-8475: Fixed EL query parser when input contains multiple escaped expressions No new revisions were added by this update. Summary of changes: .../org/apache/nifi/attribute/expression/language/Query.java | 1 + .../apache/nifi/attribute/expression/language/TestQuery.java | 12 2 files changed, 13 insertions(+)
[nifi] branch main updated: NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/main by this push: new 786f171 NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail 786f171 is described below commit 786f17182b11cfd5ea28678853a4d401a51c6aff Author: exceptionfactory AuthorDate: Tue Jun 22 12:44:34 2021 -0500 NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail - Added jakarata.activation 2.0.1 to support jakarta.mail 2.0.1 - Clarified description of SMTP TLS property enabling STARTTLS NIFI-8630 Added final to several variables and adjusted variable declaration Signed-off-by: Pierre Villard This closes #5179. --- nifi-assembly/LICENSE | 4 +- nifi-assembly/NOTICE | 12 +- .../src/main/resources/META-INF/LICENSE| 316 + .../src/main/resources/META-INF/NOTICE | 24 +- .../nifi-standard-processors/pom.xml | 16 +- .../apache/nifi/processors/standard/PutEmail.java | 110 +++ .../nifi/processors/standard/TestPutEmail.java | 25 +- nifi-nar-bundles/nifi-standard-bundle/pom.xml | 21 +- 8 files changed, 435 insertions(+), 93 deletions(-) diff --git a/nifi-assembly/LICENSE b/nifi-assembly/LICENSE index 624948a..7336f34 100644 --- a/nifi-assembly/LICENSE +++ b/nifi-assembly/LICENSE @@ -2645,7 +2645,7 @@ The binary distribution of this product bundles 'normalize.css' OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -The binary distribution of this product bundles 'JUnit Platform' and 'JUnit Jupiter' modules under an EPL v2 license +The binary distribution of this product bundles 'JUnit Platform', 'JUnit Jupiter', 'Jakarta Mail', and 'Jakarta Mail API' modules under an EPL v2 license Eclipse Public License - v 2.0 @@ -3030,7 +3030,7 @@ The binary distribution of this product bundles 'Extensions on qpid-proton-j lib OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE -The binary distribution of this product bundles 'Jakarta Activation API 1.2.1' and 'Jakarta XML Binding API 2.3.2' modules under an EDL v1.0 license +The binary distribution of this product bundles 'Jakarta Activation', 'Jakarta Activation API' and 'Jakarta XML Binding API 2.3.2' modules under an EDL v1.0 license Copyright (c) 2007, Eclipse Foundation, Inc. and its licensors. diff --git a/nifi-assembly/NOTICE b/nifi-assembly/NOTICE index 7c35cd5..cce8766 100644 --- a/nifi-assembly/NOTICE +++ b/nifi-assembly/NOTICE @@ -2068,12 +2068,22 @@ The following binary components are provided under the Eclipse Public License 1. (EPLv1.0) Clojure (org.clojure:clojure:1.8.0 - http://clojure.org) +Eclipse Public License 2.0 + + +The following binary components are provided under the Eclipse Public License 2.0. + +(EPL 2.0) Jakarta Mail API (jakarta.mail:jakarta.mail-api:jar:2.0.1) +(EPL 2.0) Jakarta Mail (com.sun.mail:jakarta.mail:jar:2.0.1) + + Eclipse Distribution License 1.0 The following binary components are provided under the Eclipse Distribution License 1.0. -(EDL 1.0) Jakarta Activation API (jakarta.activation:jakarta.activation-api:jar:1.2.1) +(EDL 1.0) Jakarta Activation API (jakarta.activation:jakarta.activation-api:jar:2.0.1) +(EDL 1.0) Jakarta Activation (com.sun.activation:jakarta.activation:jar:2.0.1) (EDL 1.0) Jakarta XML Binding API (jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2) * diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/LICENSE b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/LICENSE index 7fd9a34..b17cf20 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/LICENSE +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/LICENSE @@ -378,3 +378,319 @@ This product bundles 'SLF4J-API 1.7.21' which is available under an MIT license. OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +This product bundles 'Jakarta Activation' 'Jakarta Activation API' under the Eclipse Distribution License 1.0. + +Copyright (c) 2007, Eclipse Foundation, Inc. and its licensors. + +All rights reserved. + +Redistribution and use in source and binary forms, with or +without modification, are permitted provided that the following +conditions are met: + +- Redistributions of source code must retain the above copyright + notice, this list of
[nifi] branch main updated: NIFI-8699 - Adding the lucene-backward-codecs dependency to maintain backward compatibility with old provenance repos.
This is an automated email from the ASF dual-hosted git repository.
exceptionfactory pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 1dc528a NIFI-8699 - Adding the lucene-backward-codecs dependency to
maintain backward compatibility with old provenance repos.
1dc528a is described below
commit 1dc528ae80f15d87f439117fd7928ff89af22932
Author: Nathan Gough
AuthorDate: Thu Jun 24 13:36:54 2021 -0400
NIFI-8699 - Adding the lucene-backward-codecs dependency to maintain
backward compatibility with old provenance repos.
This closes #5187
Signed-off-by: David Handermann
---
.../nifi-persistent-provenance-repository/pom.xml| 4
nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml | 5 +
2 files changed, 9 insertions(+)
diff --git
a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml
b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml
index 5708a2a..3d83217 100644
---
a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml
+++
b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml
@@ -63,6 +63,10 @@
lucene-queryparser
+org.apache.lucene
+lucene-backward-codecs
+
+
org.apache.commons
commons-lang3
test
diff --git a/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml
b/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml
index 85a2bd5..d6b25ac 100644
--- a/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml
+++ b/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml
@@ -57,6 +57,11 @@
${lucene.version}
+org.apache.lucene
+lucene-backward-codecs
+${lucene.version}
+
+
org.apache.commons
commons-lang3
3.9
[nifi] branch main updated (1313ee3 -> 88cc232)
This is an automated email from the ASF dual-hosted git repository. mcgilman pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git. from 1313ee3 NIFI-8735 - Increase polling frequency for NAR autoloading providers (#5183) add 88cc232 NIFI-8731: If a processor is running but made invalid due to a parameter update, it ends up in a state of STARTING. If the parameter is then updated again, it transitions the state to STOPPING but the processor is not fully stopped yet. At that point, the parameter is updated and the processor is attempted to be started again. Fixed this by keeping the number of active threads to >= 1 if processor is STOPPING in order to convey that it is not fully stopped. Also addresse [...] No new revisions were added by this update. Summary of changes: .../nifi/controller/StandardProcessorNode.java | 30 +++-- .../apache/nifi/groups/StandardProcessGroup.java | 2 +- .../apache/nifi/reporting/AbstractEventAccess.java | 4 +- .../apache/nifi/web/api/FlowUpdateResource.java| 4 +- .../nifi/web/api/ParameterContextResource.java | 2 +- .../util/ClusterReplicationComponentLifecycle.java | 51 +++-- .../nifi/web/util/LocalComponentLifecycle.java | 52 +++--- .../scheduling/StatelessProcessScheduler.java | 2 +- .../system/parameters/ParameterContextIT.java | 27 +++ 9 files changed, 134 insertions(+), 40 deletions(-)
[nifi] branch main updated (49d10db -> 1313ee3)
This is an automated email from the ASF dual-hosted git repository. denes pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git. from 49d10db NIFI-5573 Allow overriding of nifi-env.sh add 1313ee3 NIFI-8735 - Increase polling frequency for NAR autoloading providers (#5183) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/nifi/nar/NarAutoLoader.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[nifi] branch main updated: NIFI-5573 Allow overriding of nifi-env.sh
This is an automated email from the ASF dual-hosted git repository.
kdoran pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 49d10db NIFI-5573 Allow overriding of nifi-env.sh
49d10db is described below
commit 49d10dbcbd584075f9cfb0d04772fb553277c7da
Author: Lars Francke
AuthorDate: Tue Sep 4 15:30:42 2018 +0200
NIFI-5573 Allow overriding of nifi-env.sh
This closes #2985.
Signed-off-by: Kevin Doran
---
.../src/main/resources/bin/nifi-env.sh | 37 ++
1 file changed, 31 insertions(+), 6 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi-env.sh
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi-env.sh
index 5889d49..4922351 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi-env.sh
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi-env.sh
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/usr/bin/env sh
#
#Licensed to the Apache Software Foundation (ASF) under one or more
#contributor license agreements. See the NOTICE file distributed with
@@ -16,16 +16,39 @@
#limitations under the License.
#
+# By default this file will unconditionally override whatever environment
variables you have set
+# and set them to defaults defined here.
+# If you want to define your own versions outside of this script please set
the environment variable
+# NIFI_OVERRIDE_NIFIENV to "true". That will then use whatever variables you
used outside of
+# this script.
+
# The java implementation to use.
#export JAVA_HOME=/usr/java/jdk1.8.0/
-export NIFI_HOME=$(cd "${SCRIPT_DIR}" && cd .. && pwd)
+setOrDefault() {
+ envvar="$1"
+ default="$2"
+
+ res="$default"
+ if [ -n "$envvar" ] && [ "$NIFI_OVERRIDE_NIFIENV" = "true" ]
+ then
+res="$envvar"
+ fi
+
+ echo "$res"
+}
+
+
+NIFI_HOME="$(setOrDefault "$NIFI_HOME" "$(cd "$SCRIPT_DIR" && cd .. && pwd)")"
+export NIFI_HOME
#The directory for the NiFi pid file
-export NIFI_PID_DIR="${NIFI_HOME}/run"
+NIFI_PID_DIR="$(setOrDefault "$NIFI_PID_DIR" "$NIFI_HOME/run")"
+export NIFI_PID_DIR
#The directory for NiFi log files
-export NIFI_LOG_DIR="${NIFI_HOME}/logs"
+NIFI_LOG_DIR="$(setOrDefault "$NIFI_LOG_DIR" "$NIFI_HOME/logs")"
+export NIFI_LOG_DIR
# Set to false to force the use of Keytab controller service in processors
# that use Kerberos. If true, these processors will allow configuration of
keytab
@@ -34,8 +57,10 @@ export NIFI_LOG_DIR="${NIFI_HOME}/logs"
# a multi-tenant environment where management of keytabs should be performed
only by
# a user with elevated permissions (i.e., users that have been granted the
'ACCESS_KEYTAB'
# restriction).
-export NIFI_ALLOW_EXPLICIT_KEYTAB=true
+NIFI_ALLOW_EXPLICIT_KEYTAB="$(setOrDefault "$NIFI_ALLOW_EXPLICIT_KEYTAB" true)"
+export NIFI_ALLOW_EXPLICIT_KEYTAB
# Set to true to deny access to the Local File System from HDFS Processors
# This flag forces HDFS Processors to evaluate the File System path during
scheduling
-export NIFI_HDFS_DENY_LOCAL_FILE_SYSTEM_ACCESS=false
\ No newline at end of file
+NIFI_HDFS_DENY_LOCAL_FILE_SYSTEM_ACCESS="$(setOrDefault
"$NIFI_HDFS_DENY_LOCAL_FILE_SYSTEM_ACCESS" false)"
+export NIFI_HDFS_DENY_LOCAL_FILE_SYSTEM_ACCESS
[nifi] branch main updated: NIFI-8737: Fixed incorrect provenance data in HDFS processors when Directory property is inconsistent with core-site.xml
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 39ffa4a NIFI-8737: Fixed incorrect provenance data in HDFS processors
when Directory property is inconsistent with core-site.xml
39ffa4a is described below
commit 39ffa4a8ac21b83f0b269828d4b162131e393903
Author: Peter Turcsanyi
AuthorDate: Wed Jun 23 17:04:08 2021 +0200
NIFI-8737: Fixed incorrect provenance data in HDFS processors when
Directory property is inconsistent with core-site.xml
---
.../processors/hadoop/AbstractHadoopProcessor.java | 23 +
.../nifi/processors/hadoop/AbstractPutHDFS.java| 25 +
.../processors/hadoop/AbstractFetchHDFSRecord.java | 2 +-
.../processors/hadoop/AbstractPutHDFSRecord.java | 3 +-
.../apache/nifi/processors/hadoop/DeleteHDFS.java | 2 +-
.../apache/nifi/processors/hadoop/FetchHDFS.java | 2 +-
.../org/apache/nifi/processors/hadoop/GetHDFS.java | 4 +-
.../nifi/processors/hadoop/GetHDFSFileInfo.java| 3 +-
.../apache/nifi/processors/hadoop/ListHDFS.java| 4 +-
.../apache/nifi/processors/hadoop/MoveHDFS.java| 21
.../org/apache/nifi/processors/hadoop/PutHDFS.java | 12 ++---
.../nifi/processors/hadoop/AbstractHadoopTest.java | 59 ++
12 files changed, 117 insertions(+), 43 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
b/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
index 1cce996..2838a38 100644
---
a/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
+++
b/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
@@ -34,6 +34,7 @@ import
org.apache.nifi.components.resource.ResourceCardinality;
import org.apache.nifi.components.resource.ResourceReferences;
import org.apache.nifi.components.resource.ResourceType;
import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.hadoop.KerberosProperties;
import org.apache.nifi.hadoop.SecurityUtil;
import org.apache.nifi.kerberos.KerberosCredentialsService;
@@ -606,4 +607,26 @@ public abstract class AbstractHadoopProcessor extends
AbstractProcessor {
}
}
+protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor property) {
+return getNormalizedPath(context, property, null);
+}
+
+protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor property, FlowFile flowFile) {
+final String propertyValue =
context.getProperty(property).evaluateAttributeExpressions(flowFile).getValue();
+final Path path = new Path(propertyValue);
+final URI uri = path.toUri();
+
+final URI fileSystemUri = getFileSystem().getUri();
+
+if (uri.getScheme() != null) {
+if (!uri.getScheme().equals(fileSystemUri.getScheme()) ||
!uri.getAuthority().equals(fileSystemUri.getAuthority())) {
+getLogger().warn("The filesystem component of the URI
configured in the '{}' property ({}) does not match the filesystem URI from the
Hadoop configuration file ({}) " +
+"and will be ignored.", property.getDisplayName(),
uri, fileSystemUri);
+}
+
+return new Path(uri.getPath());
+} else {
+return path;
+}
+}
}
diff --git
a/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractPutHDFS.java
b/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractPutHDFS.java
index 44fae65..b86b7dc 100644
---
a/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractPutHDFS.java
+++
b/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractPutHDFS.java
@@ -100,13 +100,12 @@ public abstract class AbstractPutHDFS extends
AbstractHadoopProcessor {
Path tempDotCopyFile = null;
FlowFile putFlowFile = flowFile;
try {
-final String dirValue =
context.getProperty(DIRECTORY).evaluateAttributeExpressions(putFlowFile).getValue();
-final Path configuredRootDirPath = new Path(dirValue);
+final Path dirPath = getNormalizedPath(context, DIRECTORY,
putFlowFile);
final String conflictResponse =
context.getProperty(CONFLICT_RESOLUTION).getValue();
-final
