[jira] [Created] (NIFIREG-399) Docker in docker failing for nifi docker maven

[Nihar Sanjay Kininge (Jira)]

Nihar Sanjay Kininge created NIFIREG-399:


 Summary: Docker in docker failing for nifi docker maven
 Key: NIFIREG-399
 URL: https://issues.apache.org/jira/browse/NIFIREG-399
 Project: NiFi Registry
  Issue Type: Bug
Reporter: Nihar Sanjay Kininge


dockermaven in nifi_registry failed to expand variables UID and GID while 
running build inside docker container.

`COPY --chown=${UID}:${GID}` command failing form same.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] MikeThomsen opened a new pull request #4389: NIFI-7604 Moved nifi-elasticsearch-5 NAR to a profile to save space.

[GitBox]

MikeThomsen opened a new pull request #4389:
URL: https://github.com/apache/nifi/pull/4389


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Enables X functionality; fixes bug NIFI-._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] MikeThomsen commented on a change in pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data

[GitBox]

MikeThomsen commented on a change in pull request #4370:
URL: https://github.com/apache/nifi/pull/4370#discussion_r450574794



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -321,6 +328,15 @@ public void process(final InputStream in) throws 
IOException {
 
attributes.put(CoreAttributes.ABSOLUTE_PATH.key(), absPathString);
 attributes.put(CoreAttributes.MIME_TYPE.key(), 
OCTET_STREAM);
 
+attributes.put(FILE_INNER_PERMISSION, 
String.valueOf(tarEntry.getMode()));

Review comment:
   Wouldn't storing it as, say `755`, make it easier for downstream users 
to directly convert it to a filesystem permission?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] markap14 commented on a change in pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…

[GitBox]

markap14 commented on a change in pull request #4374:
URL: https://github.com/apache/nifi/pull/4374#discussion_r450517533



##
File path: 
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
##
@@ -0,0 +1,391 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.script;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyDescriptor.Builder;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.script.ScriptingComponentHelper;
+import org.apache.nifi.script.ScriptingComponentUtils;
+import org.apache.nifi.search.SearchContext;
+import org.apache.nifi.search.SearchResult;
+import org.apache.nifi.search.Searchable;
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.WriteResult;
+import org.apache.nifi.serialization.record.Record;
+import org.python.jsr223.PyScriptEngine;
+
+import javax.script.Bindings;
+import javax.script.CompiledScript;
+import javax.script.ScriptContext;
+import javax.script.ScriptEngine;
+import javax.script.ScriptException;
+import javax.script.SimpleBindings;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+
+@EventDriven
+@SupportsBatching
+@SideEffectFree
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"record", "transform", "script", "groovy", "jython", "python", 
"update", "modify", "filter"})
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Sets the 
mime.type attribute to the MIME Type specified by the Record Writer"),
+@WritesAttribute(attribute = "record.count", description = "The number of 
records in the FlowFile"),
+@WritesAttribute(attribute = "record.error.message", description = "This 
attribute provides on failure the error message encountered by the Reader or 
Writer.")
+})
+@CapabilityDescription("Provides the ability to evaluate a simple script 
against each record in an incoming FlowFile. The script may transform the 
record in some way, filter the record, or fork " +
+"additional records. See Processor's Additional Details for more 
information.")
+@SeeAlso(classNames = {"org.apache.nifi.processors.script.ExecuteScript",
+"org.apache.nifi.processors.standard.UpdateRecord",
+"org.apache.nifi.processors.standard.QueryRecord",
+"org.apache.nifi.processors.standard.JoltTransformRecord",
+"org.apache.nifi.processors.standard.LookupRecord"})
+public class ScriptedTransformRecord extends 

[GitHub] [nifi] alopresto commented on pull request #4377: NIFI-7568 - Fixing Kerberos mappings

[GitBox]

alopresto commented on pull request #4377:
URL: https://github.com/apache/nifi/pull/4377#issuecomment-654499917


   I believe @mtien-apache has also made some changes around the OIDC provider 
in [PR 4344](https://github.com/apache/nifi/pull/4344). 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (NIFI-7605) Make InvokeHttp not send User-Agent header as default behavior

[Mike Thomsen (Jira)]

Mike Thomsen created NIFI-7605:
--

 Summary: Make InvokeHttp not send User-Agent header as default 
behavior
 Key: NIFI-7605
 URL: https://issues.apache.org/jira/browse/NIFI-7605
 Project: Apache NiFi
  Issue Type: Task
Reporter: Mike Thomsen
Assignee: Mike Thomsen






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (NIFI-7604) Move nifi-elasticsearch5-processors and NAR package to profile build

[Mike Thomsen (Jira)]

Mike Thomsen created NIFI-7604:
--

 Summary: Move nifi-elasticsearch5-processors and NAR package to 
profile build
 Key: NIFI-7604
 URL: https://issues.apache.org/jira/browse/NIFI-7604
 Project: Apache NiFi
  Issue Type: Task
Reporter: Mike Thomsen
Assignee: Mike Thomsen


General sense on the mailing list was that we can deprecate this to a profile 
build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] dazhitchman commented on pull request #4385: NIFI-7600 - Defining JMS attribute types breaks due to invalid charac…

[GitBox]

dazhitchman commented on pull request #4385:
URL: https://github.com/apache/nifi/pull/4385#issuecomment-654467047


   Hi Joey
   
   I think that opens up others issues. NIFI uses attributes for both data (to 
augment content) and metadata to control behaviour.  In the JMS example, there 
is no valid JMS field for “JMS_IBM_Encoding.type” , it is just used to help the 
processor build the headers so it shouldn’t leak from the processor.  The mime 
type however is used by downstream applications like a web/email server so they 
should be passed along. I am new to NIFI though so happy to take advice.
   
   Darren
   
   > On 7 Jul 2020, at 5:23 am, Joey  wrote:
   > 
   > 
   > @jfrazee commented on this pull request.
   > 
   > In 
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java
 :
   > 
   > > @@ -108,6 +108,13 @@ void setMessageHeaderAndProperties(final Session 
session, final Message message,
   >  }
   >  } else {
   >  // not a special attribute handled above, so send 
it as a property using the specified property type
   > +String key=entry.getKey();
   > +if (key.endsWith(".type")){
   > If we go broad and apply this to anything ending in '.type' I think it 
deserves a WARN.
   > 
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub 
, or 
unsubscribe 
.
   > 
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] turcsanyip commented on a change in pull request #4371: NIFI-7589 Fix path value when unpacking tar

[GitBox]

turcsanyip commented on a change in pull request #4371:
URL: https://github.com/apache/nifi/pull/4371#discussion_r450468675



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -309,7 +309,10 @@ public void process(final InputStream in) throws 
IOException {
 }
 final File file = new File(tarEntry.getName());
 final Path filePath = file.toPath();
-final String filePathString = filePath.getParent() 
+ "/";
+String filePathString = "/";

Review comment:
   You could avoid variable reassignment by moving it to the `else` case of 
the if statement.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] turcsanyip commented on a change in pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data

[GitBox]

turcsanyip commented on a change in pull request #4370:
URL: https://github.com/apache/nifi/pull/4370#discussion_r450430244



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -321,6 +328,15 @@ public void process(final InputStream in) throws 
IOException {
 
attributes.put(CoreAttributes.ABSOLUTE_PATH.key(), absPathString);
 attributes.put(CoreAttributes.MIME_TYPE.key(), 
OCTET_STREAM);
 
+attributes.put(FILE_INNER_PERMISSION, 
String.valueOf(tarEntry.getMode()));

Review comment:
   The permissions should be added in "rwx" format instead of an integer.

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -321,6 +328,15 @@ public void process(final InputStream in) throws 
IOException {
 
attributes.put(CoreAttributes.ABSOLUTE_PATH.key(), absPathString);
 attributes.put(CoreAttributes.MIME_TYPE.key(), 
OCTET_STREAM);
 
+attributes.put(FILE_INNER_PERMISSION, 
String.valueOf(tarEntry.getMode()));
+attributes.put(FILE_INNER_OWNER, 
String.valueOf(tarEntry.getUserName()));
+attributes.put(FILE_INNER_GROUP, 
String.valueOf(tarEntry.getGroupName()));
+
+String timePattern = "-MM-dd'T'HH:mm:ssZ";
+DateFormat df = new 
SimpleDateFormat(timePattern);

Review comment:
   Instead of `SimpleDateFormat`, `DateTimeFormatter` should be used in 
Java 8+ which is thread safe and can be defined as a constant.
   The built-in `DateTimeFormatter.ISO_OFFSET_DATE_TIME` instance seems to me 
the right format here.

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -110,6 +112,11 @@
 
 public static final String OCTET_STREAM = "application/octet-stream";
 
+public static final String FILE_INNER_PERMISSION = "file.inner.permission";
+public static final String FILE_INNER_OWNER = "file.inner.owner";
+public static final String FILE_INNER_GROUP = "file.inner.group";
+public static final String FILE_INNER_LAST_MODIFIED_TIME = 
"file.inner.lastModifiedTime";

Review comment:
   In my opinion the regular `file.*` attributes should be used and there's 
no need to introduce the "inner" ones.
   The properties of the source tar file will be preserved on the flowfile 
transferred to the `original` relationship and I don't think they are needed on 
the unpacked files.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] jfrazee commented on a change in pull request #4385: NIFI-7600 - Defining JMS attribute types breaks due to invalid charac…

[GitBox]

jfrazee commented on a change in pull request #4385:
URL: https://github.com/apache/nifi/pull/4385#discussion_r450432161



##
File path: 
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java
##
@@ -108,6 +108,13 @@ void setMessageHeaderAndProperties(final Session session, 
final Message message,
 }
 } else {
 // not a special attribute handled above, so send it 
as a property using the specified property type
+String key=entry.getKey();
+if (key.endsWith(".type")){

Review comment:
   If we go broad and apply this to anything ending in '.type' I think it 
deserves a WARN.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] jfrazee commented on pull request #4385: NIFI-7600 - Defining JMS attribute types breaks due to invalid charac…

[GitBox]

jfrazee commented on pull request #4385:
URL: https://github.com/apache/nifi/pull/4385#issuecomment-654420328


   @dazhitchman Thanks for this! I was just thinking about this over the 
weekend and it's definitely needed. One question: should it be so broad as to 
remove anything ending in '.type' or would it make sense to do something more 
narrow? E.g., I always see this with `mime.type` since it frequently gets 
populated but if someone has defined their own attributes it might not be 
obvious where they're going.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-7602) Contrib Check issues do not appear to work/alert in Github Actions as currently configured

[Joe Witt (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Witt updated NIFI-7602:
---
Component/s: Tools and Build

> Contrib Check issues do not appear to work/alert in Github Actions as 
> currently configured
> --
>
> Key: NIFI-7602
> URL: https://issues.apache.org/jira/browse/NIFI-7602
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Tools and Build
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Here is an example of something that should have failed due to contrib-check 
> issues but it didnt.
> https://github.com/apache/nifi/pull/4358
> This build in particularly should have failed due to unused imports
> https://github.com/apache/nifi/pull/4358/checks?sha=d61b926ed15cb404299545f0a5b30ee08414ce74



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (NIFI-7602) Contrib Check issues do not appear to work/alert in Github Actions as currently configured

[Joe Witt (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Witt updated NIFI-7602:
---
Issue Type: Task  (was: Test)

> Contrib Check issues do not appear to work/alert in Github Actions as 
> currently configured
> --
>
> Key: NIFI-7602
> URL: https://issues.apache.org/jira/browse/NIFI-7602
> Project: Apache NiFi
>  Issue Type: Task
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Here is an example of something that should have failed due to contrib-check 
> issues but it didnt.
> https://github.com/apache/nifi/pull/4358
> This build in particularly should have failed due to unused imports
> https://github.com/apache/nifi/pull/4358/checks?sha=d61b926ed15cb404299545f0a5b30ee08414ce74



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (NIFI-7602) Contrib Check issues do not appear to work/alert in Github Actions as currently configured

[Joe Witt (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Witt updated NIFI-7602:
---
Fix Version/s: 1.12.0

> Contrib Check issues do not appear to work/alert in Github Actions as 
> currently configured
> --
>
> Key: NIFI-7602
> URL: https://issues.apache.org/jira/browse/NIFI-7602
> Project: Apache NiFi
>  Issue Type: Test
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Here is an example of something that should have failed due to contrib-check 
> issues but it didnt.
> https://github.com/apache/nifi/pull/4358
> This build in particularly should have failed due to unused imports
> https://github.com/apache/nifi/pull/4358/checks?sha=d61b926ed15cb404299545f0a5b30ee08414ce74



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] joewitt opened a new pull request #4388: NIFI-7602 added verify phase so contrib-check can actually execute

[GitBox]

joewitt opened a new pull request #4388:
URL: https://github.com/apache/nifi/pull/4388


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Enables X functionality; fixes bug NIFI-._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] bbende commented on pull request #4377: NIFI-7568 - Fixing Kerberos mappings

[GitBox]

bbende commented on pull request #4377:
URL: https://github.com/apache/nifi/pull/4377#issuecomment-654400187


   This might be outside the scope of this PR, or maybe something we don't even 
support, but I was looking at everywhere that a `LoginAuthenticationToken` is 
constructed and noticed there is one place outside of `AccessResource`, in 
`StandardOidcIdentityProvider`, should we apply the identity mappings there too 
or is it not a problem in this case?
   
   
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/oidc/StandardOidcIdentityProvider.java#L319



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers

[Mark Payne (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-7563:
-
Fix Version/s: 1.12.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> Optimize the usage of JMS sessions and message producers
> 
>
> Key: NIFI-7563
> URL: https://issues.apache.org/jira/browse/NIFI-7563
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4
>Reporter: Gardella Juan Pablo
>Assignee: Gardella Juan Pablo
>Priority: Minor
> Fix For: 1.12.0
>
>  Time Spent: 27h 20m
>  Remaining Estimate: 44h 50m
>
> Below an scenario to reproduce the non optimize usage of JMS resources. 
> Suppose it is required to publish 1 message to the destination {{D}} using 
> [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html].
>  The message is a flow file in the processor input queue.
> It is important to know that internally the processor is using 
> [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java]
>  to reuse objects and a 
> [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180]
>  to be able to use in thread safe manner. For JMS publishers, the default 
> configuration is to cache connections, sessions (only 1) and message 
> producers.
> *Preconditions*
>  # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute 
> defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. 
> Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. 
> That will limit the scenario only when {{jms_replyTo}} attribute is defined.
>  # For simplicity, the processor is the first time it processes messages.
> *Scenario*
>  # Processor picks the message. The 
> [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180]
>  is created.
>  # Connection {{C1}} and session {{S1}} are created. The 
> [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] 
> {{M1_S1}} is created and 
> [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html]
>  {{MP_S1}} created too. Required to deliver first message at 
> [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65].
>  # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching 
> connection factory is created at 
> [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208].
>  # An attempt to create a new connection and a new session are requested to 
> the connection factory to create destination defined in the header 
> {{jms_destination}} at 
> [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131].
>  Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it 
> is required to check internal logic in CachingConnectionFactory to understand 
> why not). A new session {{S2}} is created and stored in the 
> {{CachingConnectionFactory}} as the new cached session.
>  # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not 
> in the cache, it is physically closed and {{MP_S1}}.
>  # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, 
> all resources should be reused.
> The scenario if it is applied to N consecutive messages create a lot of 
> sessions and message producers. 
> We found this issue by adding an 
> [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ 
> v5.x|http://activemq.apache.org/components/classic/] broker to detect the 
> optimal usage of resources. For example, only one message producer per 
> connection. In below scenario we will be created N producers for the same 
> connection. Also in a Nifi flow that connects a 
> 

[jira] [Resolved] (NIFIREG-227) GitFlowPersistenceProvider option to clone repo on startup

[Bryan Bende (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFIREG-227.
-
Fix Version/s: 0.7.0
   Resolution: Fixed

> GitFlowPersistenceProvider option to clone repo on startup
> --
>
> Key: NIFIREG-227
> URL: https://issues.apache.org/jira/browse/NIFIREG-227
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>Assignee: Rabi Kumar K C
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: nifi-registry-app.log
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> To build on NIFIREG-209, which added the ability to rebuild the flow metadata 
> database from a git repository if the metadata is empty on startup, it would 
> also be nice to have the option to clone a git repo on startup if there is 
> not a local git repo and a remote is configured.
> The proposed feature is for the GitFlowPersistenceProvider, to check if there 
> is a local repository during startup. If the repository is not present, and a 
> remote is configured (to push to), start by cloning the remote.
> The use case is *not* supporting multiple registry instances syncing to the 
> same git repository (that would require more work). Rather, the use case is 
> recovery from a remote git repository. This will be especially useful when 
> launching new docker containers as it would give the ability to create a 
> container initialized from a remote git repo just via configuration options 
> and not an explicit git clone step in the image/container.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-registry] asfgit closed pull request #248: NIFIREG-227 GitFlowPersistenceProvider option to clone repo on startup

[GitBox]

asfgit closed pull request #248:
URL: https://github.com/apache/nifi-registry/pull/248


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] MuazmaZ commented on a change in pull request #4369: NIFI-7581 Add CS-based credential settings support for ADLS processors

[GitBox]

MuazmaZ commented on a change in pull request #4369:
URL: https://github.com/apache/nifi/pull/4369#discussion_r450387709



##
File path: 
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureDataLakeStorageProcessor.java
##
@@ -158,41 +173,51 @@
 public static Collection 
validateCredentialProperties(final ValidationContext validationContext) {
 final List results = new ArrayList<>();
 
-final boolean useManagedIdentity = 
validationContext.getProperty(USE_MANAGED_IDENTITY).asBoolean();
-final boolean accountKeyIsSet  = 
validationContext.getProperty(ACCOUNT_KEY).isSet();
-final boolean sasTokenIsSet = 
validationContext.getProperty(SAS_TOKEN).isSet();
-
-int credential_config_found = 0;
-if(useManagedIdentity) credential_config_found++;
-if(accountKeyIsSet) credential_config_found++;
-if(sasTokenIsSet) credential_config_found++;
-
-if(credential_config_found == 0){
-final String msg = String.format(
-"At least one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
-} else if(credential_config_found > 1) {
-final String msg = String.format(
-"Only one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
+if 
(!validationContext.getProperty(AzureStorageUtils.STORAGE_CREDENTIALS_SERVICE).isSet())
 {

Review comment:
   @tpalfy makes sense to me especially if that is the pattern for the 
newer processor.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-registry] bbende commented on pull request #248: NIFIREG-227 GitFlowPersistenceProvider option to clone repo on startup

[GitBox]

bbende commented on pull request #248:
URL: https://github.com/apache/nifi-registry/pull/248#issuecomment-654380777


   Sorry for taking so long to get to this... tested and everything looks good, 
going to merge



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] mtien-apache commented on pull request #4362: NIFI 7542 Upgrade additional jackson-databind versions.

[GitBox]

mtien-apache commented on pull request #4362:
URL: https://github.com/apache/nifi/pull/4362#issuecomment-654380312


   @MikeThomsen That was the last commit. Thanks.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (NIFIREG-385) Make revision feature configurable

[Bryan Bende (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFIREG-385.
-
Resolution: Fixed

> Make revision feature configurable
> --
>
> Key: NIFIREG-385
> URL: https://issues.apache.org/jira/browse/NIFIREG-385
> Project: NiFi Registry
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Major
> Fix For: 0.7.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> In order to make the master branch be compatible with current NiFi releases, 
> we should make the revision feature configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-registry] asfgit closed pull request #284: NIFIREG-385 Returning default Revision with version 0 on RevisableEnt…

[GitBox]

asfgit closed pull request #284:
URL: https://github.com/apache/nifi-registry/pull/284


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] tpalfy commented on a change in pull request #4369: NIFI-7581 Add CS-based credential settings support for ADLS processors

[GitBox]

tpalfy commented on a change in pull request #4369:
URL: https://github.com/apache/nifi/pull/4369#discussion_r450368696



##
File path: 
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureDataLakeStorageProcessor.java
##
@@ -158,41 +173,51 @@
 public static Collection 
validateCredentialProperties(final ValidationContext validationContext) {
 final List results = new ArrayList<>();
 
-final boolean useManagedIdentity = 
validationContext.getProperty(USE_MANAGED_IDENTITY).asBoolean();
-final boolean accountKeyIsSet  = 
validationContext.getProperty(ACCOUNT_KEY).isSet();
-final boolean sasTokenIsSet = 
validationContext.getProperty(SAS_TOKEN).isSet();
-
-int credential_config_found = 0;
-if(useManagedIdentity) credential_config_found++;
-if(accountKeyIsSet) credential_config_found++;
-if(sasTokenIsSet) credential_config_found++;
-
-if(credential_config_found == 0){
-final String msg = String.format(
-"At least one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
-} else if(credential_config_found > 1) {
-final String msg = String.format(
-"Only one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
+if 
(!validationContext.getProperty(AzureStorageUtils.STORAGE_CREDENTIALS_SERVICE).isSet())
 {

Review comment:
   I think it would make sense to have the credentials configurable only 
via the controller service.
   I see the newer processors follow this pattern as well.
   
   @MuazmaZ, would you be okay with removing all the Azure credentials-related 
properties from the processors and move them to the controller service?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] IlyaKovalev opened a new pull request #4387: NIFI-7603 fix doc

[GitBox]

IlyaKovalev opened a new pull request #4387:
URL: https://github.com/apache/nifi/pull/4387


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
NIFI-7603 fix doc for in 
nifi-nar-bundles\nifi-standard-services\nifi-lookup-services-bundle\nifi-lookup-services\src\main\resources\docs\org.apache.nifi.lookup.RestLookupService\additionalDetails.html
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-registry] markap14 commented on pull request #284: NIFIREG-385 Returning default Revision with version 0 on RevisableEnt…

[GitBox]

markap14 commented on pull request #284:
URL: https://github.com/apache/nifi-registry/pull/284#issuecomment-654352545


   Good catch @bbende, changes look good to me. +1



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (NIFI-7563) Optimize the usage of JMS sessions and message producers

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152151#comment-17152151
 ] 

ASF subversion and git services commented on NIFI-7563:
---

Commit e1584207d114d4915cc793d633039b724f6a07cb in nifi's branch 
refs/heads/main from Gardella Juan Pablo
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e158420 ]

NIFI-7563: Optimize the usage of JMS sessions and message producers

The introduced changes prevent creating unnecesary sessions and producers
in some scenarios.

This closes #4378.

Signed-off-by: Joey Frazee 


> Optimize the usage of JMS sessions and message producers
> 
>
> Key: NIFI-7563
> URL: https://issues.apache.org/jira/browse/NIFI-7563
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4
>Reporter: Gardella Juan Pablo
>Assignee: Gardella Juan Pablo
>Priority: Minor
>  Time Spent: 27h 20m
>  Remaining Estimate: 44h 50m
>
> Below an scenario to reproduce the non optimize usage of JMS resources. 
> Suppose it is required to publish 1 message to the destination {{D}} using 
> [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html].
>  The message is a flow file in the processor input queue.
> It is important to know that internally the processor is using 
> [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java]
>  to reuse objects and a 
> [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180]
>  to be able to use in thread safe manner. For JMS publishers, the default 
> configuration is to cache connections, sessions (only 1) and message 
> producers.
> *Preconditions*
>  # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute 
> defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. 
> Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. 
> That will limit the scenario only when {{jms_replyTo}} attribute is defined.
>  # For simplicity, the processor is the first time it processes messages.
> *Scenario*
>  # Processor picks the message. The 
> [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180]
>  is created.
>  # Connection {{C1}} and session {{S1}} are created. The 
> [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] 
> {{M1_S1}} is created and 
> [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html]
>  {{MP_S1}} created too. Required to deliver first message at 
> [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65].
>  # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching 
> connection factory is created at 
> [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208].
>  # An attempt to create a new connection and a new session are requested to 
> the connection factory to create destination defined in the header 
> {{jms_destination}} at 
> [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131].
>  Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it 
> is required to check internal logic in CachingConnectionFactory to understand 
> why not). A new session {{S2}} is created and stored in the 
> {{CachingConnectionFactory}} as the new cached session.
>  # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not 
> in the cache, it is physically closed and {{MP_S1}}.
>  # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, 
> all resources should be reused.
> The scenario if it is applied to N consecutive messages create a lot of 
> sessions and message producers. 
> We found this issue by adding an 
> [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ 
> v5.x|http://activemq.apache.org/components/classic/] broker to detect the 
> optimal usage of 

[jira] [Commented] (NIFI-7594) HadleHttpRequest doesn't clean temporary files

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152152#comment-17152152
 ] 

ASF subversion and git services commented on NIFI-7594:
---

Commit c396927299586b896df4ebc745793b4c451f3898 in nifi's branch 
refs/heads/main from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c396927 ]

NIFI-7594 In HandleHttpRequest deleting multipart file resources after 
processing.

This closes #4379.

Signed-off-by: Peter Turcsanyi 


> HadleHttpRequest doesn't clean temporary files
> --
>
> Key: NIFI-7594
> URL: https://issues.apache.org/jira/browse/NIFI-7594
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The HandleHttpRequest processor stores multipart messages in the temporary 
> directory defined by the "java.io.tmpdir" system property.
> These files remain there (until the jvm is gracefully shut down).
> As NiFi is designed to run for long periods of time we should clean up these 
> temporary files after the content has been processed and the flowfile has 
> been transferred.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (NIFI-7603) Fix RestLookupService documentation

[Ilya Kovalev (Jira)]

Ilya Kovalev created NIFI-7603:
--

 Summary: Fix RestLookupService documentation
 Key: NIFI-7603
 URL: https://issues.apache.org/jira/browse/NIFI-7603
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Ilya Kovalev


{color:#1d1c1d}nifi-nar-bundles\nifi-standard-services\nifi-lookup-services-bundle\nifi-lookup-services\src\main\resources\docs\org.apache.nifi.lookup.RestLookupService\additionalDetails.html{color}

{{replace body with request.body}}
{{i waste about 2 hour trying to understand what going wrong))}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (NIFIREG-385) Make revision feature configurable

[Bryan Bende (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende reopened NIFIREG-385:
-

Re-opening to fix a bug when deleting revisable entities.

> Make revision feature configurable
> --
>
> Key: NIFIREG-385
> URL: https://issues.apache.org/jira/browse/NIFIREG-385
> Project: NiFi Registry
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Major
> Fix For: 0.7.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> In order to make the master branch be compatible with current NiFi releases, 
> we should make the revision feature configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-registry] bbende opened a new pull request #284: NIFIREG-385 Returning default Revision with version 0 on RevisableEnt…

[GitBox]

bbende opened a new pull request #284:
URL: https://github.com/apache/nifi-registry/pull/284


   …ity when revisions are disabled, so that there is always a non-null 
revision in case it is referenced



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (MINIFICPP-1277) Create cross-platform API for querying memory usage

[Arpad Boda (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpad Boda updated MINIFICPP-1277:
--
Labels: selected-for-development  (was: )

> Create cross-platform API for querying memory usage
> ---
>
> Key: MINIFICPP-1277
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1277
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Reporter: Ádám Markovics
>Assignee: Ádám Markovics
>Priority: Major
>  Labels: selected-for-development
>
> We would like to monitor memory usage of MiNiFi process, therefore a 
> cross-platform API is a necessity. It should work for Windows, Linux and Mac. 
> Also a test is needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1280) StringUtils::replaceAll hangs in case from_string is empty

[Arpad Boda (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpad Boda updated MINIFICPP-1280:
--
Labels: selected-for-development  (was: )

> StringUtils::replaceAll hangs in case from_string is empty
> --
>
> Key: MINIFICPP-1280
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1280
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
>  Labels: selected-for-development
> Fix For: 0.8.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Thee function simply goes into an infinite loop when source string is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1280) StringUtils::replaceAll hangs in case from_string is empty

[Arpad Boda (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpad Boda resolved MINIFICPP-1280.
---
Resolution: Fixed

> StringUtils::replaceAll hangs in case from_string is empty
> --
>
> Key: MINIFICPP-1280
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1280
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
>  Labels: selected-for-development
> Fix For: 0.8.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Thee function simply goes into an infinite loop when source string is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1273) Clear connections on flow update

[Arpad Boda (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpad Boda updated MINIFICPP-1273:
--
Labels: selected-for-development  (was: )

> Clear connections on flow update
> 
>
> Key: MINIFICPP-1273
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1273
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Adam Debreceni
>Priority: Major
>  Labels: selected-for-development
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Upon a flow update all the previous connection and processors remain in 
> memory as they circularly reference each other through shared_ptr-s. This 
> wouldn't cause much of a problem as they are in the order of kb-s, but the 
> flowfiles in these connections are also leaked which could be a significant 
> sum. 
> We should drain the connections on flow shutdown. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1274) Flow restart could double-spend flowfiles

[Arpad Boda (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpad Boda updated MINIFICPP-1274:
--
Labels: selected-for-development  (was: )

> Flow restart could double-spend flowfiles
> -
>
> Key: MINIFICPP-1274
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1274
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Adam Debreceni
>Assignee: Adam Debreceni
>Priority: Major
>  Labels: selected-for-development
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Flowfiles are async-deleted from the FlowFileRepository and no flush happens 
> after shutdown, leaving these marked files in the repository. If we restart 
> the agent, this allows these zombie files to be resurrected and be put back 
> into their last connections. This could cause files to be processed multiple 
> times even if we marked them for deletion.
> Solution proposal: flush the FlowFileRepository after shutdown, so all marked 
> files are actually deleted (this won't save us from double-processing 
> flowfiles after a crash)
> (also make sure that the FlowFileRepository shutdown happens after no more 
> processors are running)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #831: MINIFICPP-1280 - StringUtils::replaceAll hangs in case from_string is…

[GitBox]

arpadboda closed pull request #831:
URL: https://github.com/apache/nifi-minifi-cpp/pull/831


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (NIFI-2072) Support named captures in ExtractText

[Otto Fowler (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152081#comment-17152081
 ] 

Otto Fowler commented on NIFI-2072:
---

That is a good question.  Usually (in my experience) when a behavior of a 
processor is changed it is put behind a configuration property, so that is the 
convention I followed.   [~pvillard] did this as well.

> Support named captures in ExtractText
> -
>
> Key: NIFI-2072
> URL: https://issues.apache.org/jira/browse/NIFI-2072
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Otto Fowler
>Priority: Major
>  Labels: extracttext
>
> ExtractText currently captures and creates attributes using numeric indices 
> (e.g, attribute.name.0, attribute.name.1, etc.) whether or not the capture 
> groups are named, i.e., patterns like (?\w+).
> In addition to being more faithful to the provided regexes, named captures 
> could help simplify data flows because you wouldn't have to add superfluous 
> UpdateAttribute steps which are just renaming the indexed captures to more 
> interpretable names.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] ottobackwards commented on a change in pull request #4384: NIFI-2702 Support named captures in ExtractText

[GitBox]

ottobackwards commented on a change in pull request #4384:
URL: https://github.com/apache/nifi/pull/4384#discussion_r450276914



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExtractText.java
##
@@ -69,14 +71,26 @@
 + "The results of those Regular Expressions are assigned to 
FlowFile Attributes.  "
 + "Regular Expressions are entered by adding user-defined 
properties; "
 + "the name of the property maps to the Attribute Name into 
which the result will be placed.  "
++ "The attributes are generated differently based on the 
enablement of named capture groups.  "

Review comment:
   fixed





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (NIFI-7602) Contrib Check issues do not appear to work/alert in Github Actions as currently configured

[Joe Witt (Jira)]

Joe Witt created NIFI-7602:
--

 Summary: Contrib Check issues do not appear to work/alert in 
Github Actions as currently configured
 Key: NIFI-7602
 URL: https://issues.apache.org/jira/browse/NIFI-7602
 Project: Apache NiFi
  Issue Type: Test
Reporter: Joe Witt
Assignee: Joe Witt


Here is an example of something that should have failed due to contrib-check 
issues but it didnt.

https://github.com/apache/nifi/pull/4358

This build in particularly should have failed due to unused imports

https://github.com/apache/nifi/pull/4358/checks?sha=d61b926ed15cb404299545f0a5b30ee08414ce74



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-minifi-cpp] adamdebreceni opened a new pull request #833: WIP: MINIFICPP-1272 - Graceful shutdown on flow update

[GitBox]

adamdebreceni opened a new pull request #833:
URL: https://github.com/apache/nifi-minifi-cpp/pull/833


   Will conflict with #822 as they both use the newly introduced test 
processors.
   
   
   
   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #832: WIP: MINIFICPP-1272 - Graceful shutdown on flow update

[GitBox]

adamdebreceni commented on pull request #832:
URL: https://github.com/apache/nifi-minifi-cpp/pull/832#issuecomment-654270480


   not based on master 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni closed pull request #832: WIP: MINIFICPP-1272 - Graceful shutdown on flow update

[GitBox]

adamdebreceni closed pull request #832:
URL: https://github.com/apache/nifi-minifi-cpp/pull/832


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni opened a new pull request #832: WIP: MINIFICPP-1272 - Graceful shutdown on flow update

[GitBox]

adamdebreceni opened a new pull request #832:
URL: https://github.com/apache/nifi-minifi-cpp/pull/832


   Will conflict with #822 as they both use the same newly introduced test 
processors.
   
   ---
   
   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] thenatog commented on a change in pull request #4377: NIFI-7568 - Fixing Kerberos mappings

[GitBox]

thenatog commented on a change in pull request #4377:
URL: https://github.com/apache/nifi/pull/4377#discussion_r450245985



##
File path: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-administration/src/main/java/org/apache/nifi/admin/service/impl/StandardKeyService.java
##
@@ -119,10 +119,18 @@ public void deleteKey(String identity) {
 
 // delete the keys
 DeleteKeysAction deleteKeys = new DeleteKeysAction(identity);
-transaction.execute(deleteKeys);
-
-// commit the transaction
-transaction.commit();
+Integer rowsDeleted = transaction.execute(deleteKeys);
+
+// commit the transaction if we found one and only one matching 
user identity
+if(rowsDeleted == 0) {
+rollback(transaction);
+throw new AdministrationException("Unable to find user 
identity " + identity + " to remove token.");
+} else if(rowsDeleted > 1) {

Review comment:
   I don't believe so, however I thought it best to provide more feedback 
from the DAO. One problem with investigating this issue was that there was not 
enough information provided at this level, and was only really identified by 
accessing the actual queries to h2. By confirming we deleted 1 row, and 
throwing exceptions for 0 or greater than 1, we can catch potential issues like 
this.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-registry] bbende edited a comment on pull request #281: NIFIREG-394: call onConfigured with the same classloader

[GitBox]

bbende edited a comment on pull request #281:
URL: https://github.com/apache/nifi-registry/pull/281#issuecomment-654256495


   @ChrisEnglert  thanks for submitting, sorry for the delay in looking at it...
   
   Since we already have a precedent for how this done on the Authorizer's 
onConfigured [1] and we also have to do this for AccessPolicyProvider, in 
addition to UserGroupProvide, can we make them all consistent with the way the 
Authorizer is already done?
   
   All three can be done like this within the existing for loops without having 
to modify the create methods:
   ```
   final ClassLoader instanceClassLoader = instance.getClass().getClassLoader();
   try (final ExtensionCloseable extClosable = 
ExtensionCloseable.withClassLoader(instanceClassLoader)) {
   
instance.onConfigured(loadAuthorizerConfiguration(provider.getIdentifier(), 
provider.getProperty()));
   }
   ```
   
   [1] 
https://github.com/apache/nifi-registry/blob/master/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/AuthorizerFactory.java#L214-L217



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] thenatog commented on a change in pull request #4377: NIFI-7568 - Fixing Kerberos mappings

[GitBox]

thenatog commented on a change in pull request #4377:
URL: https://github.com/apache/nifi/pull/4377#discussion_r450245985



##
File path: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-administration/src/main/java/org/apache/nifi/admin/service/impl/StandardKeyService.java
##
@@ -119,10 +119,18 @@ public void deleteKey(String identity) {
 
 // delete the keys
 DeleteKeysAction deleteKeys = new DeleteKeysAction(identity);
-transaction.execute(deleteKeys);
-
-// commit the transaction
-transaction.commit();
+Integer rowsDeleted = transaction.execute(deleteKeys);
+
+// commit the transaction if we found one and only one matching 
user identity
+if(rowsDeleted == 0) {
+rollback(transaction);
+throw new AdministrationException("Unable to find user 
identity " + identity + " to remove token.");
+} else if(rowsDeleted > 1) {

Review comment:
   I don't believe so, however I thought it best to provide more feedback 
from the DAO. One problem with investigating this issue was that there was not 
enough information provided at this level, and was only really identified by 
accessing the actual queries to h2. By throwing exceptions for 0 or greater 
than 1, we can catch potential issues like this.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #831: MINIFICPP-1280 - StringUtils::replaceAll hangs in case from_string is…

[GitBox]

fgerlits commented on a change in pull request #831:
URL: https://github.com/apache/nifi-minifi-cpp/pull/831#discussion_r450239801



##
File path: extensions/windows-event-log/ConsumeWindowsEventLog.cpp
##
@@ -572,6 +572,9 @@ bool ConsumeWindowsEventLog::createEventRender(EVT_HANDLE 
hEvent, EventRender& e
 
   for (const auto  : walker.getIdentifiers()) {
 // replace the identifiers with their translated strings.
+if (mapEntry.first.empty() || mapEntry.second.empty()) {
+  continue;  // This is most probably a result of a failed ID 
resolution
+}

Review comment:
   I would move this check to the top of `StringUtils::replaceAll()`, as a 
`"" -> ""` replacement should always be a no-op, and there is no reason to 
iterate over a long string character-by-character in these cases.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-registry] bbende commented on pull request #281: NIFIREG-394: call onConfigured with the same classloader

[GitBox]

bbende commented on pull request #281:
URL: https://github.com/apache/nifi-registry/pull/281#issuecomment-654256495


   @ChrisEnglert  thanks for submitting, sorry for the delay in looking at it...
   
   Since we already have a precedent for how this done on the Authorizer's 
onConfigured [1] and we also have to do this for AccessPolicyProvider, in 
addition to UserGroupProvide, can we make them all consistent with the way the 
Authorizer is already done?
   
   All three can be this with in the existing for loops without having to 
modify the create methods:
   ```
   final ClassLoader instanceClassLoader = instance.getClass().getClassLoader();
   try (final ExtensionCloseable extClosable = 
ExtensionCloseable.withClassLoader(instanceClassLoader)) {
   
instance.onConfigured(loadAuthorizerConfiguration(provider.getIdentifier(), 
provider.getProperty()));
   }
   ```
   
   [1] 
https://github.com/apache/nifi-registry/blob/master/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/AuthorizerFactory.java#L214-L217



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #831: MINIFICPP-1280 - StringUtils::replaceAll hangs in case from_string is…

[GitBox]

fgerlits commented on a change in pull request #831:
URL: https://github.com/apache/nifi-minifi-cpp/pull/831#discussion_r450239801



##
File path: extensions/windows-event-log/ConsumeWindowsEventLog.cpp
##
@@ -572,6 +572,9 @@ bool ConsumeWindowsEventLog::createEventRender(EVT_HANDLE 
hEvent, EventRender& e
 
   for (const auto  : walker.getIdentifiers()) {
 // replace the identifiers with their translated strings.
+if (mapEntry.first.empty() || mapEntry.second.empty()) {
+  continue;  // This is most probably a result of a failed ID 
resolution
+}

Review comment:
   I would move this check to the top of `StringUtils::replaceAll()`, as a 
`"" -> ""` replacement should always be a no-op, and there is no reason to 
potentially iterate over a long string character-by-character in these cases.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] gerdansantos edited a comment on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logica

[GitBox]

gerdansantos edited a comment on pull request #4065:
URL: https://github.com/apache/nifi/pull/4065#issuecomment-654248439


   > @davyam ,
   > Thank you so much for a quick and clear update!
   > But I will post noobie question then...Is it possible to build this 
processor separately and deploy it to already running NiFi instance?
   Hello @yegorbond , 
   please download this NAR from my drive: 
https://1drv.ms/u/s!Aj33huQbOqCJhaUUH7IpGlZYBhQcoQ?e=w10Pb1
   
   after put this NAR file at $NIFI_HOME_DIR/lib/ 
   so restart your nifi.
   
   Let me know if you have some questions about this processor.
   
   I took advantage and attached the NAR in this answer.
   
   
[nifi-cdc-postgresql-nar-1.11.0-SNAPSHOT.zip](https://github.com/apache/nifi/files/4879281/nifi-cdc-postgresql-nar-1.11.0-SNAPSHOT.zip)
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] gerdansantos commented on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Repli

[GitBox]

gerdansantos commented on pull request #4065:
URL: https://github.com/apache/nifi/pull/4065#issuecomment-654248439


   > @davyam ,
   > Thank you so much for a quick and clear update!
   > But I will post noobie question then...Is it possible to build this 
processor separately and deploy it to already running NiFi instance?
   Hello @yegorbond , 
   please download this NAR from my drive: 
https://1drv.ms/u/s!Aj33huQbOqCJhaUUH7IpGlZYBhQcoQ?e=w10Pb1
   
   after put this NAR file at $NIFI_HOME_DIR/lib/ 
   so restart your nifi.
   
   Let me know if you have some questions about this processor.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] yegorbond commented on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Replicat

[GitBox]

yegorbond commented on pull request #4065:
URL: https://github.com/apache/nifi/pull/4065#issuecomment-654241021


   @davyam ,
   Thank you so much for a quick and clear update!
   But I will post noobie question then...Is it possible to build this 
processor separately and deploy it to already running NiFi instance?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] arpadboda opened a new pull request #831: MINIFICPP-1280 - StringUtils::replaceAll hangs in case from_string is…

[GitBox]

arpadboda opened a new pull request #831:
URL: https://github.com/apache/nifi-minifi-cpp/pull/831


   … empty
   
   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (NIFIREG-368) Registry breaks when key password and keystore password differ

[Bryan Bende (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFIREG-368.
-
Fix Version/s: 0.7.0
   Resolution: Fixed

> Registry breaks when key password and keystore password differ
> --
>
> Key: NIFIREG-368
> URL: https://issues.apache.org/jira/browse/NIFIREG-368
> Project: NiFi Registry
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Justin Rittenhouse
>Assignee: Nathan Gough
>Priority: Major
> Fix For: 0.7.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> (Running via Docker)
> If nifi.registry.security.keystorePasswd and nifi.registry.security.keyPasswd 
> differ, the registry fails to boot.  Running via Docker, the container shuts 
> down.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (NIFI-7594) HadleHttpRequest doesn't clean temporary files

[Peter Turcsanyi (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Turcsanyi updated NIFI-7594:
--
Component/s: Extensions

> HadleHttpRequest doesn't clean temporary files
> --
>
> Key: NIFI-7594
> URL: https://issues.apache.org/jira/browse/NIFI-7594
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The HandleHttpRequest processor stores multipart messages in the temporary 
> directory defined by the "java.io.tmpdir" system property.
> These files remain there (until the jvm is gracefully shut down).
> As NiFi is designed to run for long periods of time we should clean up these 
> temporary files after the content has been processed and the flowfile has 
> been transferred.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (NIFI-7594) HadleHttpRequest doesn't clean temporary files

[Peter Turcsanyi (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Turcsanyi resolved NIFI-7594.
---
Fix Version/s: 1.12.0
   Resolution: Fixed

> HadleHttpRequest doesn't clean temporary files
> --
>
> Key: NIFI-7594
> URL: https://issues.apache.org/jira/browse/NIFI-7594
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The HandleHttpRequest processor stores multipart messages in the temporary 
> directory defined by the "java.io.tmpdir" system property.
> These files remain there (until the jvm is gracefully shut down).
> As NiFi is designed to run for long periods of time we should clean up these 
> temporary files after the content has been processed and the flowfile has 
> been transferred.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-registry] bbende merged pull request #282: NIFIREG-368 - Fixed a transposition of the key password and keystore …

[GitBox]

bbende merged pull request #282:
URL: https://github.com/apache/nifi-registry/pull/282


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (NIFI-7594) HadleHttpRequest doesn't clean temporary files

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152005#comment-17152005
 ] 

ASF subversion and git services commented on NIFI-7594:
---

Commit c396927299586b896df4ebc745793b4c451f3898 in nifi's branch 
refs/heads/master from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c396927 ]

NIFI-7594 In HandleHttpRequest deleting multipart file resources after 
processing.

This closes #4379.

Signed-off-by: Peter Turcsanyi 


> HadleHttpRequest doesn't clean temporary files
> --
>
> Key: NIFI-7594
> URL: https://issues.apache.org/jira/browse/NIFI-7594
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The HandleHttpRequest processor stores multipart messages in the temporary 
> directory defined by the "java.io.tmpdir" system property.
> These files remain there (until the jvm is gracefully shut down).
> As NiFi is designed to run for long periods of time we should clean up these 
> temporary files after the content has been processed and the flowfile has 
> been transferred.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-registry] bbende commented on pull request #282: NIFIREG-368 - Fixed a transposition of the key password and keystore …

[GitBox]

bbende commented on pull request #282:
URL: https://github.com/apache/nifi-registry/pull/282#issuecomment-654222595


   Looks good, going to merge



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] asfgit closed pull request #4379: NIFI-7594 Clean temporary files in HandleHttpRequest

[GitBox]

asfgit closed pull request #4379:
URL: https://github.com/apache/nifi/pull/4379


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] davyam edited a comment on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Repl

[GitBox]

davyam edited a comment on pull request #4065:
URL: https://github.com/apache/nifi/pull/4065#issuecomment-653900456


   > Hello team!
   > Do you have any plans to release this PR in the nearest time?
   > 
   > Thanks!
   
   Hi @yegorbond,
   
   Saying as PR developers, we wish that this PR be included in the next NiFi 
version.
   
   This PR can help a lot of people.
   
   ASAP we should make more adjusts based on the lastest @mattyb149 comments.
   
   But, we believe that these lastest adjusts are improvements, so they can be 
done in another JIRA for future versions.
   
   Besides that, it can take a while because @gerdansantos and I have a lot of 
work to do in our professional jobs.
   
   Thanks for the support.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (MINIFICPP-1280) StringUtils::replaceAll hangs in case from_string is empty

[Arpad Boda (Jira)]

Arpad Boda created MINIFICPP-1280:
-

 Summary: StringUtils::replaceAll hangs in case from_string is empty
 Key: MINIFICPP-1280
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1280
 Project: Apache NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 0.7.0
Reporter: Arpad Boda
Assignee: Arpad Boda
 Fix For: 0.8.0


Thee function simply goes into an infinite loop when source string is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] MikeThomsen commented on a change in pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data

[GitBox]

MikeThomsen commented on a change in pull request #4370:
URL: https://github.com/apache/nifi/pull/4370#discussion_r450190144



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestUnpackContent.java
##
@@ -70,10 +70,14 @@ public void testTar() throws IOException {
 autoUnpackRunner.assertTransferCount(UnpackContent.REL_FAILURE, 0);
 
 final List unpacked = 
unpackRunner.getFlowFilesForRelationship(UnpackContent.REL_SUCCESS);
+
 for (final MockFlowFile flowFile : unpacked) {
 final String filename = 
flowFile.getAttribute(CoreAttributes.FILENAME.key());
 final String folder = 
flowFile.getAttribute(CoreAttributes.PATH.key());
 final Path path = dataPath.resolve(folder).resolve(filename);
+assertEquals(flowFile.getAttribute("file.inner.permission"),"420");
+assertEquals("jmcarey", flowFile.getAttribute("file.inner.owner"));
+assertEquals("mkpasswd", 
flowFile.getAttribute("file.inner.group"));

Review comment:
   Should have a test here for the last modified time attribute. You should 
be able to just use another `SimpleDateFormat` to see if you can successfully 
parse the attribute back into a `java.util.Date`.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] MikeThomsen commented on a change in pull request #4371: NIFI-7589 Fix path value when unpacking tar

[GitBox]

MikeThomsen commented on a change in pull request #4371:
URL: https://github.com/apache/nifi/pull/4371#discussion_r450189107



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -309,7 +309,10 @@ public void process(final InputStream in) throws 
IOException {
 }
 final File file = new File(tarEntry.getName());
 final Path filePath = file.toPath();
-final String filePathString = filePath.getParent() 
+ "/";
+String filePathString = "/";
+if(filePath.getParent() != null) {
+filePathString = filePath.getParent() + "/";

Review comment:
   The `/` should be moved to the front of the string concatenation for 
consistency and so that we don't have trailing separators. Once you do that, we 
can merge.

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -309,7 +309,10 @@ public void process(final InputStream in) throws 
IOException {
 }
 final File file = new File(tarEntry.getName());
 final Path filePath = file.toPath();
-final String filePathString = filePath.getParent() 
+ "/";
+String filePathString = "/";
+if(filePath.getParent() != null) {

Review comment:
   Should have a whitespace between the if and parenthesis.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] MikeThomsen commented on pull request #4362: NIFI 7542 Upgrade additional jackson-databind versions.

[GitBox]

MikeThomsen commented on pull request #4362:
URL: https://github.com/apache/nifi/pull/4362#issuecomment-654187342


   @mtien-apache LGTM. Do you have any remaining commits you need to make?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-7591) Allow PutS3Object to post to AWS Snowball

[Peter Turcsanyi (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Turcsanyi updated NIFI-7591:
--
Status: Patch Available  (was: In Progress)

> Allow PutS3Object to post to AWS Snowball
> -
>
> Key: NIFI-7591
> URL: https://issues.apache.org/jira/browse/NIFI-7591
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Juan C. Sequeiros
>Assignee: Peter Turcsanyi
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When posting using PutS3Object to AWS SNOWBALL [1]it fails with "
> In short it supports the AWS s3 API in a limited function [2]
> [1][https://aws.amazon.com/getting-started/projects/migrate-petabyte-scale-data/faq/]
> [2][https://docs.aws.amazon.com/snowball/latest/developer-guide/using-adapter.html]
> {code:java}
> Chunk encoding is not supported yet (Service: Amazon S3; Status Code: 501; 
> Error Code: NotImplemented; Request ID: null; S3 Extended Request ID: null)   
>       at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674)
>          at 
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656)
>          at 
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520)        
>  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4705) 
>         at 
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4652)     
>     at 
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1807)  
>        at 
> org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:504) 
>         at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2212)
>          at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2180)
>          at 
> org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:443) 
>         at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>          at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
>          at 
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
>          at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>          at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)      
>    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)   
>       at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>          at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>          at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>          at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>          at java.lang.Thread.run(Thread.java:748) {code}
> {code}
> AWS docs state this: [2]
> "If your solution uses the AWS SDK for Java version 1.11.0 or newer, you must 
> use the following S3ClientOptions" 
> {code:java}
> disableChunkedEncoding() – Indicates that chunked encoding is not supported 
> with the adapter.
> setPathStyleAccess(true) – Configures the adapter to use path-style access 
> for all requests.
> [1]https://aws.amazon.com/getting-started/projects/migrate-petabyte-scale-data/faq/
> [2]https://docs.aws.amazon.com/snowball/latest/developer-guide/using-adapter.html{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] turcsanyip opened a new pull request #4386: NIFI-7591: Allow PutS3Object to post to AWS Snowball

[GitBox]

turcsanyip opened a new pull request #4386:
URL: https://github.com/apache/nifi/pull/4386


   Added properties to enable/disable chunked encoding and path-style access
   for endpoints that do not support chunked encoding / only support path-style 
access.
   
   https://issues.apache.org/jira/browse/NIFI-7591
   
   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Enables X functionality; fixes bug NIFI-._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adam-markovics opened a new pull request #830: MINIFICPP-1276 - CAPITests should not leave any trash after running

[GitBox]

adam-markovics opened a new pull request #830:
URL: https://github.com/apache/nifi-minifi-cpp/pull/830


   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #829: WIP: MINIFICPP-1279 - FlowFile shouldn't inherit from Connectable

[GitBox]

adamdebreceni commented on a change in pull request #829:
URL: https://github.com/apache/nifi-minifi-cpp/pull/829#discussion_r450121956



##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -152,7 +153,10 @@ void FlowFileRepository::prune_stored_flowfiles() {
 // we find the connection for the persistent flowfile, create the 
flowfile and enqueue that
 std::shared_ptr flow_file_ref = 
std::static_pointer_cast(eventRead);
 eventRead->setStoredToRepository(true);
-search->second->put(eventRead);
+auto connection = std::static_pointer_cast(search->second);

Review comment:
   if #807 is merged Processors can also be FlowFile containers (like 
`BinFiles`) and those files should also be restored





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #829: WIP: MINIFICPP-1279 - FlowFile shouldn't inherit from Connectable

[GitBox]

adamdebreceni commented on a change in pull request #829:
URL: https://github.com/apache/nifi-minifi-cpp/pull/829#discussion_r450121956



##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -152,7 +153,10 @@ void FlowFileRepository::prune_stored_flowfiles() {
 // we find the connection for the persistent flowfile, create the 
flowfile and enqueue that
 std::shared_ptr flow_file_ref = 
std::static_pointer_cast(eventRead);
 eventRead->setStoredToRepository(true);
-search->second->put(eventRead);
+auto connection = std::static_pointer_cast(search->second);

Review comment:
   if the #807 is merged Processors might also be FlowFile containers (like 
`BinFiles`) and those files should also be restored





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

[GitBox]

hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r450110731



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,212 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+  "Input FlowFile has not exceeded the configured maximum retry count, pass 
this relationship back to the input Processor to create a limited feedback 
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+  "Input FlowFile has exceeded the configured maximum retry count, do not pass 
this relationship back to the input Processor to terminate the limited feedback 
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry 
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback 
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+  setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+  });
+  setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+  });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context, 
core::ProcessSessionFactory* /* sessionFactory */) {
+  context->getProperty(RetryAttribute.getName(), retry_attribute_);
+  context->getProperty(MaximumRetries.getName(), maximum_retries_);
+  context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+  context->getProperty(FailOnNonNumericalOverwrite.getName(), 
fail_on_non_numerical_overwrite_);
+  context->getProperty(ReuseMode.getName(), 

[jira] [Assigned] (NIFI-6128) UnpackContent has no awareness of file permissions

[Jira]

 [ 
https://issues.apache.org/jira/browse/NIFI-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tamás Bunth reassigned NIFI-6128:
-

Assignee: Tamás Bunth

> UnpackContent has no awareness of file permissions
> --
>
> Key: NIFI-6128
> URL: https://issues.apache.org/jira/browse/NIFI-6128
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.9.0
>Reporter: Justin Rittenhouse
>Assignee: Tamás Bunth
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> MergeContent using TAR for packing allows for storing permissions in the 
> newly created tarball via the tar.permissions attribute on incoming files.  
> UnpackContent discards all permissions information on extraction.  This makes 
> it impossible to restore a file from a tarball with previously recorded 
> permissions via NiFi.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450097829



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   this is a response to a deleted comment:
`text file` is a file format that has the magic number `\n` at the end of 
the file 勞
   
   (I don't know how to resurrect the comment)
   (although I didn't mean to imply that `\n` before EOF -> is text file, e.g. 
having a file with only the magic number won't make a file a zip file)





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450097829



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   this is a response to a deleted comment:
`text file` is a file format that has the magic number `\n` at the end of 
the file 勞
   
   (I don't know how to resurrect the comment)
   (although I didn't mean to imply that '\n' before EOF -> is text file, e.g. 
having a file with only the magic number won't make a file a zip file)





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450097829



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   this is a response to a deleted comment about how `text file` is a file 
format with the magic number `\n` at the end of the file (I don't know how to 
resurrect the comment)





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

szaszm commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450075979



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   I'm not sure about the above implication (`\n` before EOF -> is text 
file), but its contrary (no `\n` before EOF -> is not a text file) seems to be 
true to me.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450073786



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   `text file` is a file format that has the magic number `\n` at the end 
of the file 勞 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450073786



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   `text-file` is a file format that has the magic number `\n` at the end 
of the file 勞 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] turcsanyip commented on a change in pull request #4369: NIFI-7581 Add CS-based credential settings support for ADLS processors

[GitBox]

turcsanyip commented on a change in pull request #4369:
URL: https://github.com/apache/nifi/pull/4369#discussion_r450057671



##
File path: 
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureDataLakeStorageProcessor.java
##
@@ -158,41 +173,51 @@
 public static Collection 
validateCredentialProperties(final ValidationContext validationContext) {
 final List results = new ArrayList<>();
 
-final boolean useManagedIdentity = 
validationContext.getProperty(USE_MANAGED_IDENTITY).asBoolean();
-final boolean accountKeyIsSet  = 
validationContext.getProperty(ACCOUNT_KEY).isSet();
-final boolean sasTokenIsSet = 
validationContext.getProperty(SAS_TOKEN).isSet();
-
-int credential_config_found = 0;
-if(useManagedIdentity) credential_config_found++;
-if(accountKeyIsSet) credential_config_found++;
-if(sasTokenIsSet) credential_config_found++;
-
-if(credential_config_found == 0){
-final String msg = String.format(
-"At least one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
-} else if(credential_config_found > 1) {
-final String msg = String.format(
-"Only one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
+if 
(!validationContext.getProperty(AzureStorageUtils.STORAGE_CREDENTIALS_SERVICE).isSet())
 {

Review comment:
   When the CS defined, the processor level credential properties should 
not be defined. It should be validated.
   Another option would be to get rid of the processor level credential 
properties. That would make the validation easier and the whole code simpler.

##
File path: 
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureDataLakeStorageProcessor.java
##
@@ -158,41 +173,51 @@
 public static Collection 
validateCredentialProperties(final ValidationContext validationContext) {
 final List results = new ArrayList<>();
 
-final boolean useManagedIdentity = 
validationContext.getProperty(USE_MANAGED_IDENTITY).asBoolean();
-final boolean accountKeyIsSet  = 
validationContext.getProperty(ACCOUNT_KEY).isSet();
-final boolean sasTokenIsSet = 
validationContext.getProperty(SAS_TOKEN).isSet();
-
-int credential_config_found = 0;
-if(useManagedIdentity) credential_config_found++;
-if(accountKeyIsSet) credential_config_found++;
-if(sasTokenIsSet) credential_config_found++;
-
-if(credential_config_found == 0){
-final String msg = String.format(
-"At least one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
-} else if(credential_config_found > 1) {
-final String msg = String.format(
-"Only one of ['%s', '%s', '%s'] should be set",
-ACCOUNT_KEY.getDisplayName(),
-SAS_TOKEN.getDisplayName(),
-USE_MANAGED_IDENTITY.getDisplayName()
-);
-results.add(new ValidationResult.Builder().subject("Credentials 
config").valid(false).explanation(msg).build());
+if 
(!validationContext.getProperty(AzureStorageUtils.STORAGE_CREDENTIALS_SERVICE).isSet())
 {
+final boolean useManagedIdentity = 
validationContext.getProperty(USE_MANAGED_IDENTITY).asBoolean();
+final boolean accountKeyIsSet = 
validationContext.getProperty(ACCOUNT_KEY).isSet();
+final boolean sasTokenIsSet = 
validationContext.getProperty(SAS_TOKEN).isSet();
+
+int credential_config_found = 0;
+if (useManagedIdentity) credential_config_found++;
+if (accountKeyIsSet) credential_config_found++;
+if (sasTokenIsSet) credential_config_found++;
+
+if (credential_config_found == 0) {
+final String msg = String.format(
+"At least one of ['%s', '%s', '%s'] should be set",
+ACCOUNT_KEY.getDisplayName(),
+SAS_TOKEN.getDisplayName(),
+ 

[jira] [Resolved] (MINIFICPP-1266) Set configure_ member field on ProcessContext construction

[Adam Hunyadi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Hunyadi resolved MINIFICPP-1266.
-
Resolution: Fixed

> Set configure_ member field on ProcessContext construction
> --
>
> Key: MINIFICPP-1266
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1266
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Adam Hunyadi
>Priority: Trivial
> Fix For: 0.8.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> *Background:*
> In ProcessContext there is dependence on the configure_ member field being 
> set on the public API (see 
> [ProcessContext::initializeContentRepository|https://github.com/apache/nifi-minifi-cpp/blob/master/libminifi/include/core/ProcessContext.h#L209]).
> *Proposal:*
> We should set this field in the constructor to refer to the passed argument.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1256) Run modernize-equals-default on MiNiFi and apply suggestions

[Adam Hunyadi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Hunyadi resolved MINIFICPP-1256.
-
Resolution: Done

> Run modernize-equals-default on MiNiFi and apply suggestions
> 
>
> Key: MINIFICPP-1256
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1256
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Adam Hunyadi
>Priority: Trivial
> Fix For: 0.8.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> *Background:*
> = default; is a syntax we can use to improve the readibility of the current 
> codebase.
> *Proposal:*
> Run clang-tidy and apply recommended fixes. It might generate some false 
> positives that need the developers attention.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

szaszm commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450048892



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   The tools were written to behave decently with non-text files and 
consider the line number to be the encountered `\n`s + 1. Historical posix text 
processing utilities may ignore the contents after the last `\n`, but I haven't 
seen modern tools do so. [1]
   
   Please note that I'm by no means a POSIX lawyer, and the part about handling 
is mostly speculation.
   
   [1] https://unix.stackexchange.com/a/18789/366131





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-7601) QueryRecord failing to handle arrays in 1.11.4

[Dirk Arends (Jira)]

 [ 
https://issues.apache.org/jira/browse/NIFI-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dirk Arends updated NIFI-7601:
--
Description: 
I am running into an issue QueryRecord processors on Nifi 1.11.4 which was not 
an issue on 1.11.0. If a record contains arrays of “complex” data I am not able 
to perform any queries on the record, even if the queries don’t require data 
from within the array.
  
{code:java}
2020-07-06 16:43:53,902 ERROR [Timer-Driven Process Thread-8] 
o.a.nifi.processors.standard.QueryRecord 
QueryRecord[id=01731003-15bb-12a6-7e03-8c418ab7fb97] Unable to query 
StandardFlowFileRecord[uuid=04ed077b-0556-4dda-836e-62c5f9a5c772,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1594014641476-1, container=default, 
section=1], offset=51521, 
length=334],offset=0,name=04ed077b-0556-4dda-836e-62c5f9a5c772,size=334] due to 
java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, id: 
: java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, 
id: 
java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, id: 

at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1346)
at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299)
at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352)
at 
org.apache.calcite.avatica.AvaticaResultSet.getArray(AvaticaResultSet.java:729)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.getDataType(ResultSetRecordSet.java:186)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.createSchema(ResultSetRecordSet.java:156)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.(ResultSetRecordSet.java:60)
at 
org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:332)
at 
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2746)
at 
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:325)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
[https://github.com/apache/calcite-avatica/blob/master/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L1346]

 

I believe this to be a regression, possibly introduced by this commit (though I 
haven't been able to confirm):

 

[https://github.com/apache/nifi/commit/98f9b7c033a8d80ddf43aa77f63107703077d297]

https://issues.apache.org/jira/browse/NIFI-7095

 

Example 1: Succeeds when performing a query in a QueryRecord

 

Data:
{code:java}
{"total":5,"payload":["Test"]}{code}
 

Schema:
{code:java}
{"type":"record","name":"ApiResponseData","namespace":"com.example.api","fields":[
  {"name":"total","type":"int"},
  {"name":"payload","type":[
{"type":"array","items":"string"}
  ]}
]}
{code}
 

Example 2: Fails when performing a query in a QueryRecord

 

Data:
{code:java}
{"total":5,"payload":[{"type":"Test"}]}{code}
 

Schema:
{code:java}
{"type":"record","name":"ApiResponse","namespace":"com.example.api","fields":[
  {"name":"total","type":"int"},
  {"name":"payload","type":[
"null",

{"type":"array","items":{"type":"record","name":"ApiResponseData","namespace":"com.example.api","fields":[
  {"name":"type","type":"string"}
]}}
  ]}
]}
{code}
 

I have set up a simple flow.xml which demonstrates the issue. It is 
reproducible with a fresh copy of the convenience binaries, with no special 
steps.

  was:
I am running into an issue QueryRecord processors on Nifi 1.11.4 which was not 
an issue on 1.11.0. If a record contains arrays of “complex” data I am not able 
to perform any queries on the record, even if the queries don’t require data 
from within the array.
 
{code:java}
2020-07-06 16:43:53,902 ERROR [Timer-Driven Process Thread-8] 
o.a.nifi.processors.standard.QueryRecord 
QueryRecord[id=01731003-15bb-12a6-7e03-8c418ab7fb97] Unable to query 

[jira] [Created] (NIFI-7601) QueryRecord failing to handle arrays in 1.11.4

[Dirk Arends (Jira)]

Dirk Arends created NIFI-7601:
-

 Summary: QueryRecord failing to handle arrays in 1.11.4
 Key: NIFI-7601
 URL: https://issues.apache.org/jira/browse/NIFI-7601
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.11.4
Reporter: Dirk Arends
 Attachments: flow.xml

I am running into an issue QueryRecord processors on Nifi 1.11.4 which was not 
an issue on 1.11.0. If a record contains arrays of “complex” data I am not able 
to perform any queries on the record, even if the queries don’t require data 
from within the array.
 
{code:java}
2020-07-06 16:43:53,902 ERROR [Timer-Driven Process Thread-8] 
o.a.nifi.processors.standard.QueryRecord 
QueryRecord[id=01731003-15bb-12a6-7e03-8c418ab7fb97] Unable to query 
StandardFlowFileRecord[uuid=04ed077b-0556-4dda-836e-62c5f9a5c772,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1594014641476-1, container=default, 
section=1], offset=51521, 
length=334],offset=0,name=04ed077b-0556-4dda-836e-62c5f9a5c772,size=334] due to 
java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, id: 
: java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, 
id: 
java.lang.IllegalStateException: Unhandled ARRAY component type: OBJECT, id: 

at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1346)
at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299)
at 
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352)
at 
org.apache.calcite.avatica.AvaticaResultSet.getArray(AvaticaResultSet.java:729)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.getDataType(ResultSetRecordSet.java:186)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.createSchema(ResultSetRecordSet.java:156)
at 
org.apache.nifi.serialization.record.ResultSetRecordSet.(ResultSetRecordSet.java:60)
at 
org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:332)
at 
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2746)
at 
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:325)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
[https://github.com/apache/calcite-avatica/blob/master/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L1346]

 

I believe this to be a regression, possibly introduced by this commit (though I 
haven't been able to confirm):

 

[https://github.com/apache/nifi/commit/98f9b7c033a8d80ddf43aa77f63107703077d297]

https://issues.apache.org/jira/browse/NIFI-7095

 

Example 1: Succeeds when performing a query in a QueryRecord

 

Data: 
{code:java}
{"total":5,"payload":["Test"]}{code}
 

 

Schema:

 
{code:java}
{"type":"record","name":"ApiResponseData","namespace":"com.example.api","fields":[
  {"name":"total","type":"int"},
  {"name":"payload","type":[
{"type":"array","items":"string"}
  ]}
]}
{code}
 

 

Example 2: Fails when performing a query in a QueryRecord

 

Data:
{code:java}
{"total":5,"payload":[{"type":"Test"}]}{code}
 

Schema:
{code:java}
{"type":"record","name":"ApiResponse","namespace":"com.example.api","fields":[
  {"name":"total","type":"int"},
  {"name":"payload","type":[
"null",

{"type":"array","items":{"type":"record","name":"ApiResponseData","namespace":"com.example.api","fields":[
  {"name":"type","type":"string"}
]}}
  ]}
]}
{code}
 

I have set up a simple flow.xml which demonstrates the issue. It is 
reproducible with a fresh copy of the convenience binaries, with no special 
steps.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (NIFI-2072) Support named captures in ExtractText

[Malthe Borch (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151827#comment-17151827
 ] 

Malthe Borch edited comment on NIFI-2072 at 7/6/20, 7:20 AM:
-

Is it really necessary to _enable_ named capture group rather than just use 
them? If I don't want a named capture group, I suppose I am just not going to 
name them, opting instead for enumerated ones.


was (Author: malthe):
Is it really necessary to _enable _named capture group rather than just use 
them? If I don't want a named capture group, I suppose I am just not going to 
name them, opting instead for enumerated ones.

> Support named captures in ExtractText
> -
>
> Key: NIFI-2072
> URL: https://issues.apache.org/jira/browse/NIFI-2072
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Otto Fowler
>Priority: Major
>  Labels: extracttext
>
> ExtractText currently captures and creates attributes using numeric indices 
> (e.g, attribute.name.0, attribute.name.1, etc.) whether or not the capture 
> groups are named, i.e., patterns like (?\w+).
> In addition to being more faithful to the provided regexes, named captures 
> could help simplify data flows because you wouldn't have to add superfluous 
> UpdateAttribute steps which are just renaming the indexed captures to more 
> interpretable names.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (NIFI-2072) Support named captures in ExtractText

[Malthe Borch (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151827#comment-17151827
 ] 

Malthe Borch edited comment on NIFI-2072 at 7/6/20, 7:20 AM:
-

Is it really necessary to _enable _named capture group rather than just use 
them? If I don't want a named capture group, I suppose I am just not going to 
name them, opting instead for enumerated ones.


was (Author: malthe):
Is it really necessary to enable named capture group rather than just use them? 
If I don't want a named capture group, I suppose I am just not going to name 
them, opting instead for enumerated ones.

> Support named captures in ExtractText
> -
>
> Key: NIFI-2072
> URL: https://issues.apache.org/jira/browse/NIFI-2072
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Otto Fowler
>Priority: Major
>  Labels: extracttext
>
> ExtractText currently captures and creates attributes using numeric indices 
> (e.g, attribute.name.0, attribute.name.1, etc.) whether or not the capture 
> groups are named, i.e., patterns like (?\w+).
> In addition to being more faithful to the provided regexes, named captures 
> could help simplify data flows because you wouldn't have to add superfluous 
> UpdateAttribute steps which are just renaming the indexed captures to more 
> interpretable names.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (NIFI-2072) Support named captures in ExtractText

[Malthe Borch (Jira)]

[ 
https://issues.apache.org/jira/browse/NIFI-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151827#comment-17151827
 ] 

Malthe Borch commented on NIFI-2072:


Is it really necessary to enable named capture group rather than just use them? 
If I don't want a named capture group, I suppose I am just not going to name 
them, opting instead for enumerated ones.

> Support named captures in ExtractText
> -
>
> Key: NIFI-2072
> URL: https://issues.apache.org/jira/browse/NIFI-2072
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Otto Fowler
>Priority: Major
>  Labels: extracttext
>
> ExtractText currently captures and creates attributes using numeric indices 
> (e.g, attribute.name.0, attribute.name.1, etc.) whether or not the capture 
> groups are named, i.e., patterns like (?\w+).
> In addition to being more faithful to the provided regexes, named captures 
> could help simplify data flows because you wouldn't have to add superfluous 
> UpdateAttribute steps which are just renaming the indexed captures to more 
> interpretable names.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [nifi] malthe commented on a change in pull request #4384: NIFI-2702 Support named captures in ExtractText

[GitBox]

malthe commented on a change in pull request #4384:
URL: https://github.com/apache/nifi/pull/4384#discussion_r450028255



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExtractText.java
##
@@ -69,14 +71,26 @@
 + "The results of those Regular Expressions are assigned to 
FlowFile Attributes.  "
 + "Regular Expressions are entered by adding user-defined 
properties; "
 + "the name of the property maps to the Attribute Name into 
which the result will be placed.  "
++ "The attributes are generated differently based on the 
enablement of named capture groups.  "

Review comment:
   Enabling?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450014693



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   (some more bikeshedding questions about the POSIX "line")
   
   https://user-images.githubusercontent.com/64783590/86563354-85c2ea80-bf64-11ea-8f3a-a5c2ac5f66f8.png;>
   
   what does POSIX say about where the cursor would be? it cannot be on `line 
2` as there is no `line 2`
   
   or take the error message:
   
   https://user-images.githubusercontent.com/64783590/86563648-097cd700-bf65-11ea-87cb-d8d9ebf12ddd.png;>
   
   what is the `2` in `test.cpp:2:8`?
   
   we could have "the line `n`" mean the set of characters after the `(n-1)th 
line` up to the next `\n`, but that seems like a huge stretch





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450014693



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   (some more bikeshedding questions about the POSIX "line")
   
   https://user-images.githubusercontent.com/64783590/86563354-85c2ea80-bf64-11ea-8f3a-a5c2ac5f66f8.png;>
   
   what does POSIX say about where the cursor would be? it cannot be on `line 
2` as there is no `line 2`
   
   or take the error message:
   
   https://user-images.githubusercontent.com/64783590/86563648-097cd700-bf65-11ea-87cb-d8d9ebf12ddd.png;>
   
   what is the `2` in `test.cpp:2:8`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450014693



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   (some more bikeshedding questions about the POSIX "line")
   
   https://user-images.githubusercontent.com/64783590/86563354-85c2ea80-bf64-11ea-8f3a-a5c2ac5f66f8.png;>
   
   what does POSIX say about where the cursor would be? it cannot be on "line 
2" as there is no "line 2" 
   
   or take the error message:
   
   https://user-images.githubusercontent.com/64783590/86563648-097cd700-bf65-11ea-87cb-d8d9ebf12ddd.png;>
   
   what is the `2` in `test.cpp:2:8`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #827: MINIFICPP-1273 - Drain connections on flow shutdown

[GitBox]

adamdebreceni commented on a change in pull request #827:
URL: https://github.com/apache/nifi-minifi-cpp/pull/827#discussion_r450014693



##
File path: libminifi/test/flow-tests/CMakeLists.txt
##
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+file(GLOB FLOW_TESTS  "*.cpp")
+SET(FLOW_TEST_COUNT 0)
+FOREACH(testfile ${FLOW_TESTS})
+get_filename_component(testfilename "${testfile}" NAME_WE)
+add_executable("${testfilename}" "${testfile}")
+createTests("${testfilename}")
+target_link_libraries(${testfilename} ${CATCH_MAIN_LIB})
+target_wholearchive_library(${testfilename} minifi-standard-processors)
+MATH(EXPR FLOW_TEST_COUNT "${FLOW_TEST_COUNT}+1")
+add_test(NAME "${testfilename}" COMMAND "${testfilename}" 
WORKING_DIRECTORY ${TEST_DIR})
+ENDFOREACH()
+message("-- Finished building ${FLOW_TEST_COUNT} flow related test file(s)...")

Review comment:
   (some more bikeshedding questions about the POSIX "line")
   
   https://user-images.githubusercontent.com/64783590/86563354-85c2ea80-bf64-11ea-8f3a-a5c2ac5f66f8.png;>
   
   what does POSIX say about where the cursor would be? it cannot be on "line 
2" as there is no "line 2" 
   
   or take the error message:
   
   https://user-images.githubusercontent.com/64783590/86563648-097cd700-bf65-11ea-87cb-d8d9ebf12ddd.png;>
   
   what is the `2` in `test.cpp:**2**:8`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org