date:20211202

2021-12-02 Thread ASF subversion and git services (Jira)



 [ 
https://issues.apache.org/jira/browse/NIFI-9069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Handermann resolved NIFI-9069.

Fix Version/s: 1.16.0
   Resolution: Fixed

> Allow NiFi to store dataflows using Versioned Components instead of XML
> ---
>
> Key: NIFI-9069
> URL: https://issues.apache.org/jira/browse/NIFI-9069
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.16.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Currently, we have several different ways of representing flows/sub-flows. We 
> have the flow.xml.gz, which is used to store the node's flow. We have 
> Templates that store portions of flows. We have Versioned Components 
> (VersionedProcessGroup, etc.) that can store sub-flows (and with just a 
> couple of tweaks could store an entire flow easily).
> This becomes a large maintenance burden. Additionally, there are a lot of 
> improvements that could be made by switching to the Versioned Components 
> model for serializing the dataflow.
> Much of this is discussed in the NiFi 2.0 Proposal 
> (https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Proposed+Release+Goals)
> However, moving purely from XML to JSON & Versioned Components is not without 
> risk. If there were a bug in the logic, it could result in loss of the 
> dataflow. As a result, we should build this capability such that we write out 
> using both the Versioned Component api (JSON) and the existing flow.xml.gz 
> logic.
> Then, on startup, we should load from the Versioned Component model, if it 
> exists. In doing this, the transition from xml to json will be seamless by 
> just starting up - the transformation will just happen when the dataflow is 
> written out again. And this way, if there's an issue loading the dataflow 
> using Versioned Components, the .json file could be removed and the .xml file 
> used.
> We can subsequently drop support for the XML encoding.
> This means that the DataFlow that gets serialized to nodes in a cluster will 
> also transition, and we can use much more powerful capabilities like we do 
> when inheriting a Versioned Flow in order to have a node rejoin a cluster 
> even when many things have changed.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (NIFI-9069) Allow NiFi to store dataflows using Versioned Components instead of XML



[ 
https://issues.apache.org/jira/browse/NIFI-9069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452609#comment-17452609
 ] 

ASF subversion and git services commented on NIFI-9069:
---

Commit 90b39b593a0958122f950ed1b49e4098d2935d2a in nifi's branch 
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=90b39b5 ]

NIFI-9069 Changed framework dataflow serialization to support JSON

- Changed framework so that it serializes the dataflow into a VersionedDataflow 
using JSON as well as XML, and prefers the JSON representation on load, if it's 
available. This also results in the need for the cluster protocol to exchange 
its representation of the dataflow to using JSON. Rather than re-implementing 
all of the complex logic of Flow Fingerprinting, updated to just inherit the 
cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a 
new ProcessGroupSynchronizer class instead of having all of the logic within 
StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new 
StandardFlowSynchronizer that delegates to the appropriate (Xml or 
Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are 
available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, 
reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed 
issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / 
NoClassDefFoundError if the authorizations were changed and then a node 
attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, 
which is in nifi-security-utils and so so by madking nifi-security-utils a 
provided dependency of nifi-framework-api, but this doesn't work, because 
nifi-framework-api is loaded by a higher-level classloader, so the classloader 
that loads AbstractPolicyBasedAuthorizer will never have the appropriate 
classloader to provide nifi-security-utils. Addressed this by copying the code 
for creating a safe document builder from XmlUtils to 
AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter 
contexts, one inheriting from another, where neither is pre-defined in the 
existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where 
one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann 


> Allow NiFi to store dataflows using Versioned Components instead of XML
> ---
>
> Key: NIFI-9069
> URL: https://issues.apache.org/jira/browse/NIFI-9069
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Currently, we have several different ways of representing flows/sub-flows. We 
> have the flow.xml.gz, which is used to store the node's flow. We have 
> Templates that store portions of flows. We have Versioned Components 
> (VersionedProcessGroup, etc.) that can store sub-flows (and with just a 
> couple of tweaks could store an entire flow easily).
> This becomes a large maintenance burden. Additionally, there are a lot of 
> improvements that could be made by switching to the Versioned Components 
> model for serializing the dataflow.
> Much of this is discussed in the NiFi 2.0 Proposal 
> (https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Proposed+Release+Goals)
> However, moving purely from XML to JSON & Versioned Components is not without 
> risk. If there were a bug in the logic, it could result in loss of the 
> dataflow. As a result, we should build this capability such that we write out 
> using both the Versioned Component api (JSON) and the existing flow.xml.gz 
> logic.
> Then, on startup, we should load from the Versioned Component model, if it 
> exists. In doing this, the transition from xml to json will be seamless by 
> just starting up - the transformation will just happen when the dataflow is 
> written out again. And this way, if there's an issue loading the dataflow 
> using Versioned Components, the .json file could be removed and the .xml file 
> used.
> We can subsequently drop support for the XML encoding.
> This means that the DataFlow that gets serialized to nodes in a cluster will 
> also transition, and we can use much more powerful capabilities like we do 
> when inheriting

[GitHub] [nifi] exceptionfactory closed pull request #5514: NIFI-9069: Changed framework so that it serializes the dataflow into …



exceptionfactory closed pull request #5514:
URL: https://github.com/apache/nifi/pull/5514


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] bbende commented on pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module



bbende commented on pull request #5563:
URL: https://github.com/apache/nifi/pull/5563#issuecomment-984997632


   Re lombok... 
   
   Just my two-cents, but I would probably lean against using it for classes 
that we consider API classes. I would rather us maintain control over the 
structure, rather than depending on code generation. 
   
   In other cases I would say I am neutral, I think it comes down to a lot of 
personal preference in terms of how much "magic" you like. I tend to prefer 
just using the IDE to generate getters/setters/equals/toString/etc, but then 
having the full class available like normal.
   
   Re nifi-api...
   
   I view nifi-api as the highest level API we have for the overall nifi Java 
ecosystem. It has the interfaces for the extensions that are run in NiFi and 
MiNiFi Java (Processor, ControllerService, ReportingTask), as well as the 
VersionedComponent data model for registry/flow definitions.
   
   Part of me almost suggested that the component/extension manifest API could 
become part of nifi-api since it is essentially a model that can be produced 
from extensions/components that implement the interfaces in nifi-api, but I 
didn't want to complicate it too much. So I think depending on it should be ok.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Assigned] (NIFI-9438) Separate nifi-sensitive-property-provider into specific modules



 [ 
https://issues.apache.org/jira/browse/NIFI-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Handermann reassigned NIFI-9438:
--

Assignee: David Handermann

> Separate nifi-sensitive-property-provider into specific modules
> ---
>
> Key: NIFI-9438
> URL: https://issues.apache.org/jira/browse/NIFI-9438
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: David Handermann
>Priority: Major
>
> Since nifi-sensitive-property-provider has quite a few implementations now 
> and depends on various cloud provider client libraries, it would be nice to 
> split this module up into and API and implementation modules.
> A possible approach would be something like:
> {code:java}
> nifi-sensitive-property-provider-api
> nifi-sensitive-property-provider-aws
> nifi-sensitive-property-provider-azure
> nifi-sensitive-property-provider-gcp
> nifi-sensitive-property-provider-hashicorp-vault {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] bejancsaba commented on a change in pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module

2021-12-02 Thread ASF subversion and git services (Jira)



bejancsaba commented on a change in pull request #5563:
URL: https://github.com/apache/nifi/pull/5563#discussion_r761422504



##
File path: c2/c2-protocol/c2-protocol-api/pom.xml
##
@@ -0,0 +1,55 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+c2-protocol
+org.apache.nifi
+1.16.0-SNAPSHOT
+
+
+c2-protocol-api
+jar
+
+
+
+org.apache.nifi
+nifi-api

Review comment:
   Just want to make sure I understand the intention correctly. Do we want 
the c2-protocol-api to depend on nifi-api? @kevdoran mentioned that if needed 
shared classes could be extracted so both could depend on that or that was 
about something else?

##
File path: 
c2/c2-protocol/c2-protocol-api/src/main/java/org/apache/nifi/c2/protocol/api/AgentInfo.java
##
@@ -0,0 +1,88 @@
+/*
+ * Apache NiFi - MiNiFi
+ * Copyright 2014-2018 The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.c2.protocol.api;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import java.io.Serializable;
+import javax.validation.constraints.NotBlank;
+
+@ApiModel
+public class AgentInfo implements Serializable {
+private static final long serialVersionUID = -8812289319080770084L;
+
+@NotBlank
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
+private AgentManifest agentManifest;
+private AgentStatus status;
+// private Map configProperties;  // TODO we should add 
this information eventually, but we need to handle the best way to handle 
sharing sensitive properties.

Review comment:
   Same TODO question as above but I won't comment on all TODOs just seems 
not necessary for the API definition in the current state.

##
File path: c2/pom.xml
##
@@ -0,0 +1,49 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+nifi
+org.apache.nifi
+1.16.0-SNAPSHOT
+
+
+c2
+pom
+
+
+c2-protocol
+
+
+
+
+
+org.apache.nifi
+nifi-api

Review comment:
   Just to confirm the approach, I thought the protocol-api won't depend on 
the nifi-api. I think @kevdoran mentioned that if needed shared classes could 
be extracted and both apis could depend on that or was that a suggestion for 
something else?

##
File path: 
c2/c2-protocol/c2-protocol-api/src/main/java/org/apache/nifi/c2/protocol/api/OperationType.java
##
@@ -0,0 +1,37 @@
+/*
+ * Apache NiFi - MiNiFi
+ * Copyright 2014-2018 The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.c2.protocol.api;
+
+public enum OperationType {

Review comment:
   We could update this to reflect the list in the design doc. What do you 
think? I know for the debug command the TRANSFER will be needed for sure.

##
File path: 
c2/c2-protocol/c2-protocol-api/src/main/java/org/apache/nifi/c2/protocol/api/AgentStatus.java

[jira] [Commented] (NIFI-9432) Typo in Diagnostic data - Input/Output Ports



[ 
https://issues.apache.org/jira/browse/NIFI-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452593#comment-17452593
 ] 

ASF subversion and git services commented on NIFI-9432:
---

Commit fede6b9354b7f125b908e6d9c84c47a774752008 in nifi's branch 
refs/heads/main from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fede6b9 ]

NIFI-9432 - fix typo in diagnostic output (#5562)



> Typo in Diagnostic data - Input/Output Ports
> 
>
> Key: NIFI-9432
> URL: https://issues.apache.org/jira/browse/NIFI-9432
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In the diagnostic data, we see "Input" twice when it should be Input and 
> Output:
> {code:java}
> Site-to-Site Input Ports: 0
> Site-to-Site Input Ports: 0{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9432) Typo in Diagnostic data - Input/Output Ports



 [ 
https://issues.apache.org/jira/browse/NIFI-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-9432:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Typo in Diagnostic data - Input/Output Ports
> 
>
> Key: NIFI-9432
> URL: https://issues.apache.org/jira/browse/NIFI-9432
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In the diagnostic data, we see "Input" twice when it should be Input and 
> Output:
> {code:java}
> Site-to-Site Input Ports: 0
> Site-to-Site Input Ports: 0{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] kevdoran commented on a change in pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module



kevdoran commented on a change in pull request #5563:
URL: https://github.com/apache/nifi/pull/5563#discussion_r761434158



##
File path: 
c2/c2-protocol/c2-protocol-api/src/main/java/org/apache/nifi/c2/protocol/api/AgentInfo.java
##
@@ -0,0 +1,88 @@
+/*
+ * Apache NiFi - MiNiFi
+ * Copyright 2014-2018 The Apache Software Foundation

Review comment:
   AFAIK, these copyright indicators are not necessary on every file. Only 
the license header.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] markap14 merged pull request #5562: NIFI-9432 - fix typo in diagnostic output



markap14 merged pull request #5562:
URL: https://github.com/apache/nifi/pull/5562


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] markap14 commented on pull request #5562: NIFI-9432 - fix typo in diagnostic output



markap14 commented on pull request #5562:
URL: https://github.com/apache/nifi/pull/5562#issuecomment-984973945


   Nice catch @pvillard31 . Thanks, will merge.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (NIFI-9438) Separate nifi-sensitive-property-provider into specific modules

2021-12-02 Thread Bryan Bende (Jira)

Bryan Bende created NIFI-9438:
-

 Summary: Separate nifi-sensitive-property-provider into specific 
modules
 Key: NIFI-9438
 URL: https://issues.apache.org/jira/browse/NIFI-9438
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Bryan Bende


Since nifi-sensitive-property-provider has quite a few implementations now and 
depends on various cloud provider client libraries, it would be nice to split 
this module up into and API and implementation modules.

A possible approach would be something like:
{code:java}
nifi-sensitive-property-provider-api
nifi-sensitive-property-provider-aws
nifi-sensitive-property-provider-azure
nifi-sensitive-property-provider-gcp
nifi-sensitive-property-provider-hashicorp-vault {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] mattyb149 commented on pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module



mattyb149 commented on pull request #5563:
URL: https://github.com/apache/nifi/pull/5563#issuecomment-984933687


   Yeah that makes more sense, let's divide it up by functionality, maybe an 
agent one too and a single common one they both inherit from? Then users of the 
libraries can decide which dependencies they need without including classes 
they don't want.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS

2021-12-02 Thread Matt Burgess (Jira)



 [ 
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess resolved NIFI-9436.

Fix Version/s: 1.16.0
   Resolution: Fixed

> PutParquet, PutORC processors may hang after writing to ADLS
> 
>
> Key: NIFI-9436
> URL: https://issues.apache.org/jira/browse/NIFI-9436
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Fix For: 1.16.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> h2. Background
> In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
> *org.apache.parquet.hadoop.ParquetWriter* writer is {_}created{_}, used to 
> write record to an HDFS location and is later _closed_ explicitly.
> The writer creation process involves the instantiation of an 
> *org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is 
> going to be an {*}org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem{*}.
> Note that the NiFi AbstractPutHDFSRecord processor already created a 
> FileSystem object of the same type for it's own purposes but the writer 
> creates it's own.
> It could still be the same if caching was enabled but it is explicitly 
> disabled in *AbstractHadoopProcessor* (the parent of _AbstractPutHDFSRecord_).
> The writer only uses the FileSystem object to create an 
> *org.apache.hadoop.fs.azurebfs.servicesAbfsOutputStream* object and doesn't 
> keep the FileSystem object itself.
> This makes the FileSystem object eligible for garbage collection.
> The AbfsOutputStream writes data asynchronously. Submits the task to an 
> executorservice and stores it in a collection.
> h2. The issue
>  * The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference 
> to the same _ThreadPoolExecutor_ object.
>  * _AzureBlobFileSystem_ (probably depending on the version) overrides the 
> _finalize()_ method and closes itself when that is called. This involves 
> shutting down the referenced {_}ThreadPoolExecutor{_}.
>  * It's possible for garbage collection to occur after the _ParquetWriter_ is 
> created but before explicitly closing it. GC -> 
> _AzureBlobFileSystem.finalize()_ -> {_}ThreadPoolExecutor.shutdown(){_}.
>  * When the _ParquetWriter_ is explicitly closed it tries to run a cleanup 
> job using the {_}ThreadPoolExecutor{_}. That job submission fails as the 
> _ThreadPoolExecutor_ is already terminated but a _Future_ object is still 
> created - and is being wait for indefinitely.
> This causes the processor to hang.
> h2. The solution
> This feels like an issue that should be addressed in the _hadoop-azure_ 
> library but it's possible to apply a workaround in NiFi.
> The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So 
> if the _ParquetWriter_ used the same _FileSystem_ object that the processor 
> already created for itself (and kept the reference for) it would prevent the 
> garbage collection to occur.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] mattyb149 closed pull request #5565: NIFI-9436 - In AbstractPutHDFSRecord make sure the record writers use the FileSystem object the processor already has.

2021-12-02 Thread ASF subversion and git services (Jira)



mattyb149 closed pull request #5565:
URL: https://github.com/apache/nifi/pull/5565


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS



[ 
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452570#comment-17452570
 ] 

ASF subversion and git services commented on NIFI-9436:
---

Commit ff864266f59e70a67b2a1f2c787a0f74464b6a9d in nifi's branch 
refs/heads/main from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ff86426 ]

NIFI-9436 - In AbstractPutHDFSRecord make sure the record writers use the 
FileSystem object the processor already has.

Signed-off-by: Matthew Burgess 

This closes #5565


> PutParquet, PutORC processors may hang after writing to ADLS
> 
>
> Key: NIFI-9436
> URL: https://issues.apache.org/jira/browse/NIFI-9436
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> h2. Background
> In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
> *org.apache.parquet.hadoop.ParquetWriter* writer is {_}created{_}, used to 
> write record to an HDFS location and is later _closed_ explicitly.
> The writer creation process involves the instantiation of an 
> *org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is 
> going to be an {*}org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem{*}.
> Note that the NiFi AbstractPutHDFSRecord processor already created a 
> FileSystem object of the same type for it's own purposes but the writer 
> creates it's own.
> It could still be the same if caching was enabled but it is explicitly 
> disabled in *AbstractHadoopProcessor* (the parent of _AbstractPutHDFSRecord_).
> The writer only uses the FileSystem object to create an 
> *org.apache.hadoop.fs.azurebfs.servicesAbfsOutputStream* object and doesn't 
> keep the FileSystem object itself.
> This makes the FileSystem object eligible for garbage collection.
> The AbfsOutputStream writes data asynchronously. Submits the task to an 
> executorservice and stores it in a collection.
> h2. The issue
>  * The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference 
> to the same _ThreadPoolExecutor_ object.
>  * _AzureBlobFileSystem_ (probably depending on the version) overrides the 
> _finalize()_ method and closes itself when that is called. This involves 
> shutting down the referenced {_}ThreadPoolExecutor{_}.
>  * It's possible for garbage collection to occur after the _ParquetWriter_ is 
> created but before explicitly closing it. GC -> 
> _AzureBlobFileSystem.finalize()_ -> {_}ThreadPoolExecutor.shutdown(){_}.
>  * When the _ParquetWriter_ is explicitly closed it tries to run a cleanup 
> job using the {_}ThreadPoolExecutor{_}. That job submission fails as the 
> _ThreadPoolExecutor_ is already terminated but a _Future_ object is still 
> created - and is being wait for indefinitely.
> This causes the processor to hang.
> h2. The solution
> This feels like an issue that should be addressed in the _hadoop-azure_ 
> library but it's possible to apply a workaround in NiFi.
> The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So 
> if the _ParquetWriter_ used the same _FileSystem_ object that the processor 
> already created for itself (and kept the reference for) it would prevent the 
> garbage collection to occur.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] mattyb149 commented on pull request #5565: NIFI-9436 - In AbstractPutHDFSRecord make sure the record writers use the FileSystem object the processor already has.



mattyb149 commented on pull request #5565:
URL: https://github.com/apache/nifi/pull/5565#issuecomment-984932280


   +1 LGTM, thanks for the fix! Merging to main


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] thenatog commented on a change in pull request #5560: NIFI-9321 - Updated ListenTCPRecord to use a netty server instead of …

2021-12-02 Thread ASF subversion and git services (Jira)



thenatog commented on a change in pull request #5560:
URL: https://github.com/apache/nifi/pull/5560#discussion_r761387671



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestListenTCPRecord.java
##
@@ -189,24 +198,17 @@ public void testRunClientAuthNone() throws 
InitializationException, IOException,
 Assert.assertTrue(content.contains("This is a test " + 3));
 }
 
-protected void run(final int expectedTransferred, final SSLContext 
sslContext) throws IOException, InterruptedException {
+protected void run(final int expectedTransferred, final byte[] data, final 
SSLContext sslContext, final boolean shouldInitialize) throws Exception {
 final int port = NetworkUtils.availablePort();
 runner.setProperty(ListenTCPRecord.PORT, Integer.toString(port));
 
 // Run Processor and start listener without shutting down
-runner.run(1, false, true);
-
-final Thread thread = new Thread(() -> {
-try (final Socket socket = getSocket(port, sslContext)) {
-final OutputStream outputStream = socket.getOutputStream();
-outputStream.write(DATA.getBytes(StandardCharsets.UTF_8));
-outputStream.flush();
-} catch (final IOException e) {
-LOGGER.error("Failed Sending Records to Port [{}]", port, e);
-}
-});
-thread.start();
+LOGGER.info("Before run:");
+runner.run(1, false, shouldInitialize);
+LOGGER.info("About to send messages:");
+sendMessages(port, data, sslContext);
 
+LOGGER.info("Sent messages to port: {}", port);

Review comment:
   Sorry yes, these should have been removed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (NIFI-9336) When configuring component properties, UI should flag/make obvious when there is whitespace



[ 
https://issues.apache.org/jira/browse/NIFI-9336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452548#comment-17452548
 ] 

ASF subversion and git services commented on NIFI-9336:
---

Commit 2b415de912879dc92995a482d44b7092e18f44c8 in nifi's branch 
refs/heads/main from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2b415de ]

NIFI-9336 - Show icon for property values with leading or trailing whitespace 
(#5559)

* NIFI-9336 - Show icon in processor and controller services configurations 
when property values contain leading or trailing whitespace

* - Address PR feedback

* - Fix a bug to clean up tooltips to prevent a DOM leak

This closes #5559 

> When configuring component properties, UI should flag/make obvious when there 
> is whitespace
> ---
>
> Key: NIFI-9336
> URL: https://issues.apache.org/jira/browse/NIFI-9336
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.15.0
>Reporter: Mark Payne
>Assignee: M Tien
>Priority: Major
> Fix For: 1.16.0
>
> Attachments: whitespace-awareness.png
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> It is a very common problem that users will copy & paste values into the 
> property configuration for a processor/controller service/etc. And this often 
> results in unwanted white space at the beginning or end of the property value 
> (especially the end). This can have very confusing results, like indicating 
> that a hostname cannot be found or that credentials are invalid, etc. but the 
> issue is not obvious when an error message looks like:
> {{No route to host: 1.1.1.1}}
> At the same time, we cannot simply have NiFi trim all property values, 
> because sometimes white space is important. For example, Merge Content allows 
> a header/footer/demarcator which may well intentionally include whitespace.
> While we could add an indicator to Property Descriptors to address this 
> issue, such as:
>  {{.trimWhiteSpace(true)}}
> it will be easy to forget adding that to the Property Descriptor, and if we 
> default it to true, it would change behavior of many processors unexpectedly.
> Since we cannot trim the values automatically, the UI should flag this when 
> configuring property values, making it very clear that there is leading or 
> trailing whitespace.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9336) When configuring component properties, UI should flag/make obvious when there is whitespace

2021-12-02 Thread Matt Gilman (Jira)



 [ 
https://issues.apache.org/jira/browse/NIFI-9336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-9336:
--
Fix Version/s: 1.16.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> When configuring component properties, UI should flag/make obvious when there 
> is whitespace
> ---
>
> Key: NIFI-9336
> URL: https://issues.apache.org/jira/browse/NIFI-9336
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.15.0
>Reporter: Mark Payne
>Assignee: M Tien
>Priority: Major
> Fix For: 1.16.0
>
> Attachments: whitespace-awareness.png
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> It is a very common problem that users will copy & paste values into the 
> property configuration for a processor/controller service/etc. And this often 
> results in unwanted white space at the beginning or end of the property value 
> (especially the end). This can have very confusing results, like indicating 
> that a hostname cannot be found or that credentials are invalid, etc. but the 
> issue is not obvious when an error message looks like:
> {{No route to host: 1.1.1.1}}
> At the same time, we cannot simply have NiFi trim all property values, 
> because sometimes white space is important. For example, Merge Content allows 
> a header/footer/demarcator which may well intentionally include whitespace.
> While we could add an indicator to Property Descriptors to address this 
> issue, such as:
>  {{.trimWhiteSpace(true)}}
> it will be easy to forget adding that to the Property Descriptor, and if we 
> default it to true, it would change behavior of many processors unexpectedly.
> Since we cannot trim the values automatically, the UI should flag this when 
> configuring property values, making it very clear that there is leading or 
> trailing whitespace.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] mcgilman merged pull request #5559: NIFI-9336 - Show icon for property values with leading or trailing whitespace



mcgilman merged pull request #5559:
URL: https://github.com/apache/nifi/pull/5559


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] bbende commented on pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module



bbende commented on pull request #5563:
URL: https://github.com/apache/nifi/pull/5563#issuecomment-984866056


   Awesome work putting this together!
   
   What do you think of separating the package 
`org.apache.nifi.c2.protocol.api.extension` into it's own module like 
`c2-protocol-extension-api` or `c2-protocol-component-api` ?
   
   My reasoning is that the extensions package is more of a generic model for 
describing extensions in the NiFi eco-system and could be very useful outside 
of the overall c2 protocol, where as the other stuff is more specific an agent 
sending heartbeats as part of the protocol.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] tpalfy opened a new pull request #5565: NIFI-9436 - In AbstractPutHDFSRecord make sure the record writers use the FileSystem object the processor already has.



tpalfy opened a new pull request #5565:
URL: https://github.com/apache/nifi/pull/5565


   
   
   https://issues.apache.org/jira/browse/NIFI-9436
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `main`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-9433) Load Balancer hangs



 [ 
https://issues.apache.org/jira/browse/NIFI-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-9433:
-
Assignee: Mark Payne
  Status: Patch Available  (was: Open)

> Load Balancer hangs
> ---
>
> Key: NIFI-9433
> URL: https://issues.apache.org/jira/browse/NIFI-9433
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Mark Bean
>Assignee: Mark Payne
>Priority: Critical
>  Labels: connections, load-balanced-connections, load-balancing
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Simplified scenario to demonstrate problem:
> A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
> connection -> UpdateAttribute. And, unconnected to the first two processors, 
> Funnel #1 -> non-load-balanced Connection -> Funnel #2.
> GenerateFlowFile is scheduled to run on Primary Node only. It is started. 
> This causes the connection to be very busy load balancing (round robin). 
> Then, the connection between the two funnels is removed.
> Immediately, an error is thrown, and the flow gets stuck in a state of 
> constantly throwing errors indicating that a connection (the one just 
> deleted) does not exist and cannot be balanced.
> It is unclear why this connection is being considered by the load balancer at 
> all.
> The sequence of errors include the following:
> Primary Node reports 
> 2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged 
> from FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], 
> Swap Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> 2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from 
> FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
> Files=[0], Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[206, 20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> The above may be a symptom of subsequent errors in the log:
> Primary Node reports:
> 2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
> o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 
> 
> java.io.IOException: Failed to negotiate Protocol Version with Peer 
> . Recommended version 1 but instead of an ACCEPT or REJECT 
> response got back a response of 33.
> Non-Primary Node reports:
> 2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with 
> Peer
> java.io.IOException: Expected to receive Transaction Completion Indicator 
> from Peer  but instead received a value of 1
> The highly concerning part is this error which indicates a Connection which 
> was not scheduled to load balance was attempting to receive a FlowFile.
> Non-Primary Node reports:
> 2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
> o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
> Peer  for Connection with ID  but no connection exists with that 
> ID.
> Note the that  value in this message corresponds to the Connection that 
> was removed causing the errors to begin. Should the above message ever occur? 
> Does the load balancer ever consider Connections which are configured as "Do 
> not load balance"
> Users have also reported that FlowFiles have been load balanced from one 
> Connection to another, unrelated Connection on the other Node. (This is still 
> being verified.)
> Finally, on the UI the load-balanced connection indicates it is actively load 
> balancing some number (206 in this case) of FlowFiles currently in the 
> connection. And, attempts to "list queue" on this connection show no 
> FlowFiles. Presumably they are being held by the load balancer and are 
> inaccessible in the queue.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi] markap14 opened a new pull request #5564: NIFI-9433: When a Connection is unregistered from the NioAsyncLoadBal…



markap14 opened a new pull request #5564:
URL: https://github.com/apache/nifi/pull/5564


   …anceClient, make sure that we only cancel its active transaction if the 
transaction belongs to the appropriate connection. Also ensure that when we do 
cancel a transaction / call its failure callback, we purge the collection of 
any FlowFiles that have been sent in that transaction. This ensures that we 
cannot later attempt to failure the transaction again, decrementing the count 
of FlowFiles for the connection more than we should.
   
   
   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Enables X functionality; fixes bug NIFI-._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `main`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] mattyb149 opened a new pull request #5563: NIFI-9430: Create initial C2 structure, add c2-protocol-api module



mattyb149 opened a new pull request #5563:
URL: https://github.com/apache/nifi/pull/5563


   
   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   Adds a top-level "c2" module and populates the c2-protocol-api submodule 
with API classes. Also updated MiNiFi to use the same version of Swagger that 
NiFi does. Unit tests are not included as the API classes are basically beans.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [x] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically `main`)?
   
   - [x] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [x] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] mtien-apache commented on pull request #5559: NIFI-9336 - Show icon for property values with leading or trailing whitespace



mtien-apache commented on pull request #5559:
URL: https://github.com/apache/nifi/pull/5559#issuecomment-984815961


   @mcgilman Per our offline discussion, I've pushed a commit that fixes a bug 
to clean up tooltips. With the addition of the whitespace icon, when editing a 
value cell, it was re-rendering the property table, causing property tooltips 
to become orphaned on the DOM. This no longer occurs. Can you check again? 
Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (NIFI-9437) Flowfile Expiration does not exceed 24 days

2021-12-02 Thread Tim Chermak (Jira)

Tim Chermak created NIFI-9437:
-

 Summary: Flowfile Expiration does not exceed 24 days
 Key: NIFI-9437
 URL: https://issues.apache.org/jira/browse/NIFI-9437
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Affects Versions: 1.15.0, 1.14.0
Reporter: Tim Chermak


We discovered setting a FlowFile Expiration on a queue for anything over 24 
days is ignored and not 'aged out of the flow". So, if FlowFile Expiration is 
set to anything beyond 24 days, the file does not automatically expire and 
remains in the queue. 

 

Can this be fixed so that there is no limit? Also, perhaps the Expiration 
setting can have other criteria/strategy for expiring, as indicated in an old 
ticket NiFi-372



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (NIFI-9433) Load Balancer hangs



[ 
https://issues.apache.org/jira/browse/NIFI-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452474#comment-17452474
 ] 

Mark Payne commented on NIFI-9433:
--

Thanks for reporting [~markbean]. Looking into this now.

To clarify/simplify the reproduction scenario:
- When the connection is deleted, it does not need to be between funnels. 
Doesn't matter the source/destination of the connection, just the fact that a 
connection was deleted.
- For GenerateFlowFile, set the size of the data generated to something large 
like 50 MB. This makes it more likely that there is an 'active session'.

> Load Balancer hangs
> ---
>
> Key: NIFI-9433
> URL: https://issues.apache.org/jira/browse/NIFI-9433
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Mark Bean
>Priority: Critical
>
> Simplified scenario to demonstrate problem:
> A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
> connection -> UpdateAttribute. And, unconnected to the first two processors, 
> Funnel #1 -> non-load-balanced Connection -> Funnel #2.
> GenerateFlowFile is scheduled to run on Primary Node only. It is started. 
> This causes the connection to be very busy load balancing (round robin). 
> Then, the connection between the two funnels is removed.
> Immediately, an error is thrown, and the flow gets stuck in a state of 
> constantly throwing errors indicating that a connection (the one just 
> deleted) does not exist and cannot be balanced.
> It is unclear why this connection is being considered by the load balancer at 
> all.
> The sequence of errors include the following:
> Primary Node reports 
> 2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged 
> from FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], 
> Swap Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> 2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from 
> FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
> Files=[0], Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[206, 20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> The above may be a symptom of subsequent errors in the log:
> Primary Node reports:
> 2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
> o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 
> 
> java.io.IOException: Failed to negotiate Protocol Version with Peer 
> . Recommended version 1 but instead of an ACCEPT or REJECT 
> response got back a response of 33.
> Non-Primary Node reports:
> 2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with 
> Peer
> java.io.IOException: Expected to receive Transaction Completion Indicator 
> from Peer  but instead received a value of 1
> The highly concerning part is this error which indicates a Connection which 
> was not scheduled to load balance was attempting to receive a FlowFile.
> Non-Primary Node reports:
> 2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
> o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
> Peer  for Connection with ID  but no connection exists with that 
> ID.
> Note the that  value in this message corresponds to the Connection that 
> was removed causing the errors to begin. Should the above message ever occur? 
> Does the load balancer ever consider Connections which are configured as "Do 
> not load balance"
> Users have also reported that FlowFiles have been load balanced from one 
> Connection to another, unrelated Connection on the other Node. (This is still 
> being verified.)
> Finally, on the UI the load-balanced connection indicates it is actively load 
> balancing some number (206 in this case) of FlowFiles currently in the 
> connection. And, attempts to "list queue" on this connection show no 
> FlowFiles. Presumably they are being held by the load balancer and are 
> inaccessible in the queue.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9433) Load Balancer hangs



 [ 
https://issues.apache.org/jira/browse/NIFI-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-9433:
-
Labels: connections load-balanced-connections load-balancing  (was: )

> Load Balancer hangs
> ---
>
> Key: NIFI-9433
> URL: https://issues.apache.org/jira/browse/NIFI-9433
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Mark Bean
>Priority: Critical
>  Labels: connections, load-balanced-connections, load-balancing
>
> Simplified scenario to demonstrate problem:
> A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
> connection -> UpdateAttribute. And, unconnected to the first two processors, 
> Funnel #1 -> non-load-balanced Connection -> Funnel #2.
> GenerateFlowFile is scheduled to run on Primary Node only. It is started. 
> This causes the connection to be very busy load balancing (round robin). 
> Then, the connection between the two funnels is removed.
> Immediately, an error is thrown, and the flow gets stuck in a state of 
> constantly throwing errors indicating that a connection (the one just 
> deleted) does not exist and cannot be balanced.
> It is unclear why this connection is being considered by the load balancer at 
> all.
> The sequence of errors include the following:
> Primary Node reports 
> 2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged 
> from FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], 
> Swap Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> 2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from 
> FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
> Files=[0], Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[206, 20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> The above may be a symptom of subsequent errors in the log:
> Primary Node reports:
> 2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
> o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 
> 
> java.io.IOException: Failed to negotiate Protocol Version with Peer 
> . Recommended version 1 but instead of an ACCEPT or REJECT 
> response got back a response of 33.
> Non-Primary Node reports:
> 2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with 
> Peer
> java.io.IOException: Expected to receive Transaction Completion Indicator 
> from Peer  but instead received a value of 1
> The highly concerning part is this error which indicates a Connection which 
> was not scheduled to load balance was attempting to receive a FlowFile.
> Non-Primary Node reports:
> 2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
> o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
> Peer  for Connection with ID  but no connection exists with that 
> ID.
> Note the that  value in this message corresponds to the Connection that 
> was removed causing the errors to begin. Should the above message ever occur? 
> Does the load balancer ever consider Connections which are configured as "Do 
> not load balance"
> Users have also reported that FlowFiles have been load balanced from one 
> Connection to another, unrelated Connection on the other Node. (This is still 
> being verified.)
> Finally, on the UI the load-balanced connection indicates it is actively load 
> balancing some number (206 in this case) of FlowFiles currently in the 
> connection. And, attempts to "list queue" on this connection show no 
> FlowFiles. Presumably they are being held by the load balancer and are 
> inaccessible in the queue.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (MINIFICPP-1223) Stop reloading script files every time ExecutePythonProcessor is triggered



 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gábor Gyimesi reassigned MINIFICPP-1223:


Assignee: Gábor Gyimesi  (was: Adam Hunyadi)

> Stop reloading script files every time ExecutePythonProcessor is triggered
> --
>
> Key: MINIFICPP-1223
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1223
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Adam Hunyadi
>Assignee: Gábor Gyimesi
>Priority: Minor
> Fix For: 1.0.0
>
>
> *Acceptance criteria:*
> *GIVEN* A Getfile -> ExecutePythonScript (with "Reload on Script Change" not 
> set) -> Putfile workflow
>  *WHEN* The ExecutePythonScript is run twice *with* an update on the script 
> file inbetween
>  *THEN* On the second execution the behaviour of the ExecuteScriptProcessor 
> should not change
> *GIVEN* A Getfile -> ExecutePythonScript (with "Reload on Script Change" 
> disabled) -> Putfile workflow
>  *WHEN* The ExecutePythonScript is run twice *with* an update on the script 
> file inbetween
>  *THEN* On the second execution the behaviour of the ExecuteScriptProcessor 
> should not change
> *GIVEN* A Getfile -> ExecutePythonScript (with "Reload on Script Change" 
> enabled) -> Putfile workflow
>  *WHEN* The ExecutePythonScript is run twice *with* an update on the script 
> file inbetween
>  *THEN* On the second execution the behaviour of the ExecuteScriptProcessor 
> should follow the updated script
> *Background:*
>  For backward compatibility, we went for keeping the behaviour of reading the 
> script file every time the processor is triggered intact.
> *Proposal:*
> We would like to add an option called *"Reload on Script Change"* to toggle 
> this with the first major release.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (NIFI-9433) Load Balancer hangs

2021-12-02 Thread Mark Bean (Jira)



[ 
https://issues.apache.org/jira/browse/NIFI-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452465#comment-17452465
 ] 

Mark Bean commented on NIFI-9433:
-

Still chasing this, but I'll offer what I noted. In NioAsyncLoadBalanceClient 
partitions are added to partitionQueue, but are never removed, e.g. when a 
connection is removed from the flow.

> Load Balancer hangs
> ---
>
> Key: NIFI-9433
> URL: https://issues.apache.org/jira/browse/NIFI-9433
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Mark Bean
>Priority: Critical
>
> Simplified scenario to demonstrate problem:
> A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
> connection -> UpdateAttribute. And, unconnected to the first two processors, 
> Funnel #1 -> non-load-balanced Connection -> Funnel #2.
> GenerateFlowFile is scheduled to run on Primary Node only. It is started. 
> This causes the connection to be very busy load balancing (round robin). 
> Then, the connection between the two funnels is removed.
> Immediately, an error is thrown, and the flow gets stuck in a state of 
> constantly throwing errors indicating that a connection (the one just 
> deleted) does not exist and cannot be balanced.
> It is unclear why this connection is being considered by the load balancer at 
> all.
> The sequence of errors include the following:
> Primary Node reports 
> 2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged 
> from FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], 
> Swap Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> 2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from 
> FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
> Files=[0], Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ 
> ActiveQueue=[206, 20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
> Unacknowledged=[-206, -20600 Bytes] ]
> java.lang.RuntimeException: Cannot create negative queue size
> The above may be a symptom of subsequent errors in the log:
> Primary Node reports:
> 2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
> o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 
> 
> java.io.IOException: Failed to negotiate Protocol Version with Peer 
> . Recommended version 1 but instead of an ACCEPT or REJECT 
> response got back a response of 33.
> Non-Primary Node reports:
> 2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with 
> Peer
> java.io.IOException: Expected to receive Transaction Completion Indicator 
> from Peer  but instead received a value of 1
> The highly concerning part is this error which indicates a Connection which 
> was not scheduled to load balance was attempting to receive a FlowFile.
> Non-Primary Node reports:
> 2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
> o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
> Peer  for Connection with ID  but no connection exists with that 
> ID.
> Note the that  value in this message corresponds to the Connection that 
> was removed causing the errors to begin. Should the above message ever occur? 
> Does the load balancer ever consider Connections which are configured as "Do 
> not load balance"
> Users have also reported that FlowFiles have been load balanced from one 
> Connection to another, unrelated Connection on the other Node. (This is still 
> being verified.)
> Finally, on the UI the load-balanced connection indicates it is actively load 
> balancing some number (206 in this case) of FlowFiles currently in the 
> connection. And, attempts to "list queue" on this connection show no 
> FlowFiles. Presumably they are being held by the load balancer and are 
> inaccessible in the queue.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS

[
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tamas Palfy updated NIFI-9436:
--
Description:
h2. Background

In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an
*org.apache.parquet.hadoop.ParquetWriter* writer is {_}created{_}, used to
write record to an HDFS location and is later _closed_ explicitly.

The writer creation process involves the instantiation of an
*org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is going
to be an {*}org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem{*}.
Note that the NiFi AbstractPutHDFSRecord processor already created a FileSystem
object of the same type for it's own purposes but the writer creates it's own.
It could still be the same if caching was enabled but it is explicitly disabled
in *AbstractHadoopProcessor* (the parent of _AbstractPutHDFSRecord_).

The writer only uses the FileSystem object to create an
*org.apache.hadoop.fs.azurebfs.servicesAbfsOutputStream* object and doesn't
keep the FileSystem object itself.
This makes the FileSystem object eligible for garbage collection.

The AbfsOutputStream writes data asynchronously. Submits the task to an
executorservice and stores it in a collection.
h2. The issue
* The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference to
the same _ThreadPoolExecutor_ object.
* _AzureBlobFileSystem_ (probably depending on the version) overrides the
_finalize()_ method and closes itself when that is called. This involves
shutting down the referenced {_}ThreadPoolExecutor{_}.
* It's possible for garbage collection to occur after the _ParquetWriter_ is
created but before explicitly closing it. GC ->
_AzureBlobFileSystem.finalize()_ -> {_}ThreadPoolExecutor.shutdown(){_}.
* When the _ParquetWriter_ is explicitly closed it tries to run a cleanup job
using the {_}ThreadPoolExecutor{_}. That job submission fails as the
_ThreadPoolExecutor_ is already terminated but a _Future_ object is still
created - and is being wait for indefinitely.

This causes the processor to hang.
h2. The solution

This feels like an issue that should be addressed in the _hadoop-azure_ library
but it's possible to apply a workaround in NiFi.

The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So
if the _ParquetWriter_ used the same _FileSystem_ object that the processor
already created for itself (and kept the reference for) it would prevent the
garbage collection to occur.

was:
h2. Background

This causes the processor to hang.
h2. The solution

This feels like an issue that should be addressed in the _hadoop-azure_ library
but it's possible to apply a workaround in NiFi.

The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So
if the _ParquetWriter_ used the same _FileSystem_ object that the processor
already created for itself -and kept the reference for- it would prevent the
garbage collection to occur.

> PutParquet, PutORC processors may

[jira] [Updated] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS

[
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tamas Palfy updated NIFI-9436:
--
Description:
h2. Background

This causes the processor to hang.
h2. The solution

This feels like an issue that should be addressed in the _hadoop-azure_ library
but it's possible to apply a workaround in NiFi.

The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So
if the _ParquetWriter_ used the same _FileSystem_ object that the processor
already created for itself -and kept the reference for- it would prevent the
garbage collection to occur.

was:
h2. Background
In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an
*org.apache.parquet.hadoop.ParquetWriter* writer is _created_, used to write
record to an HDFS location and is later _closed_ explicitly.

The writer creation process involves the instantiation of an
*org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is going
to be an *org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem*.
Note that the NiFi AbstractPutHDFSRecord processor already created a FileSystem
object of the same type for it's own purposes but the writer creates it's own.

The AbfsOutputStream writes data asynchronously. Submits the task to an
executorservice and stores it in a collection.

h2. The issue

* The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference to
the same _ThreadPoolExecutor_ object.
* _AzureBlobFileSystem_ (probably depending on the version) overrides the
_finalize()_ method and closes itself when that is called. This involves
shutting down the referenced _ThreadPoolExecutor_.
* It's possible for garbage collection to occur after the _ParquetWriter_ is
created but before explicitly closing it. GC ->
_AzureBlobFileSystem.finalize()_ -> _ThreadPoolExecutor.shutdown()_.
* When the _ParquetWriter_ is explicitly closed it tries to run a cleanup job
using the _ThreadPoolExecutor_. That job submission fails as the
_ThreadPoolExecutor_ is already terminated but a _Future_ object is still
created - and is being wait for indefinitely.

This causes the processor to hang.

h2. The solution

This feels like an issue that should be addressed in the _hadoop-azure_ library
but it's possible to apply a workaround in NiFi.

The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So
if the _ParquetWriter_ used the same _FileSystem_ object that the processor
already created for itself -and kept the reference for- it would prevent the
garbage collection to occur.

> PutParquet, PutORC processors may hang after writing to ADLS
>
>
> Key: NIFI-9436
> URL:

[jira] [Commented] (NIFI-9431) Issue While Migrating to 1.14.0



[ 
https://issues.apache.org/jira/browse/NIFI-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452438#comment-17452438
 ] 

David Handermann commented on NIFI-9431:


Thanks for reporting this issue [~sarahikh].  Can you provide some additional 
details? A full stack trace of the log output would be helpful.  If you are 
able to share the XML Template, or provide a sample template that creates the 
same problem, that would also help track down the problem.

> Issue While Migrating to 1.14.0 
> 
>
> Key: NIFI-9431
> URL: https://issues.apache.org/jira/browse/NIFI-9431
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Sarah Alkhudair
>Priority: Critical
>
> Dears,
> We are facing an issue in migrating from 1.13.2 to 1.14.0. When we upload 
> templates (.xml files) to 1.14.0 and then restart NiFi ,we keep having this 
> error back (WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup 
> of context o.e.j.w.WebAppContext).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS



 [ 
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tamas Palfy updated NIFI-9436:
--
Description: 
h2. Background
In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
*org.apache.parquet.hadoop.ParquetWriter* writer is _created_, used to write 
record to an HDFS location and is later _closed_ explicitly.

The writer creation process involves the instantiation of an 
*org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is going 
to be an *org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem*.
Note that the NiFi AbstractPutHDFSRecord processor already created a FileSystem 
object of the same type for it's own purposes but the writer creates it's own.

The writer only uses the FileSystem object to create an 
*org.apache.hadoop.fs.azurebfs.servicesAbfsOutputStream* object and doesn't 
keep the FileSystem object itself.
This makes the FileSystem object eligible for garbage collection.

The AbfsOutputStream writes data asynchronously. Submits the task to an 
executorservice and stores it in a collection.

h2. The issue

* The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference to 
the same _ThreadPoolExecutor_ object.
* _AzureBlobFileSystem_ (probably depending on the version) overrides the 
_finalize()_ method and closes itself when that is called. This involves 
shutting down the referenced _ThreadPoolExecutor_.
* It's possible for garbage collection to occur after the _ParquetWriter_ is 
created but before explicitly closing it. GC -> 
_AzureBlobFileSystem.finalize()_ -> _ThreadPoolExecutor.shutdown()_.
* When the _ParquetWriter_ is explicitly closed it tries to run a cleanup job 
using the _ThreadPoolExecutor_. That job submission fails as the 
_ThreadPoolExecutor_ is already terminated but a _Future_ object is still 
created -  and is being wait for indefinitely.

This causes the processor to hang.

h2. The solution

This feels like an issue that should be addressed in the _hadoop-azure_ library 
but it's possible to apply a workaround in NiFi.

The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So 
if the _ParquetWriter_ used the same _FileSystem_ object that the processor 
already created for itself -and kept the reference for- it would prevent the 
garbage collection to occur.



  was:
h2. Background
In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
*org.apache.parquet.hadoop.ParquetWriter* writer is _created_, used to write 
record to an HDFS location and is later _closed_ explicitly.

The writer creation process involves the instantiation of an 
org.apache.hadoop.fs.FileSystem object, which, when writing to ADLS is going to 
be an org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.
Note that the NiFi AbstractPutHDFSRecord processor already created a FileSystem 
object of the same type for it's own purposes but the writer creates it's own.

The writer only uses the FileSystem object to create an AbfsOutputStream object 
and doesn't keep the FileSystem object itself.
This makes the FileSystem object eligible to garbage collection.

The AbfsOutputStream writes data asynchronously. Submits the task to an 
executorservice and stores it in a collection.

h2. The issue

* The AzureBlobFileSystem and the AbfsOutputStream both have reference to the 
same ThreadPoolExecutor object.
* AzureBlobFileSystem -probably depending on the version- overrides the 
finalize() method and closes itself when that is called. This involves shutting 
down the referenced ThreadPoolExecutor.
* It's possible for garbage collection to occur after the ParquetWriter is 
created but before explicitly closing it. GC -> AzureBlobFileSystem.finalize() 
-> ThreadPoolExecutor.shutdown().
* When the ParquetWriter is explicitly closed it tries to run a cleanup job 
using the ThreadPoolExecutor. That job submission fails as the 
ThreadPoolExecutor is already terminated but a Future object is still created - 
 and is being wait for indefinitely.

This causes the processor to hang.

h2. The solution

This feels like an issue that should be addressed in the hadoop-azure library 
but it's possible to apply a workaround in NiFi.

The problem starts by the AzureBlobFileSystem getting garbage collected. So if 
the ParquetWriter used the same FileSystem object that the processor already 
created for itself -and kept the reference for- it would prevent the garbage 
collection to occur.




> PutParquet, PutORC processors may hang after writing to ADLS
> 
>
> Key: NIFI-9436
> URL: https://issues.apache.org/jira/browse/NIFI-9436
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Tamas Palfy
>Priority: Major
>
> h2. Background
> In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an

[jira] [Assigned] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS



 [ 
https://issues.apache.org/jira/browse/NIFI-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tamas Palfy reassigned NIFI-9436:
-

Assignee: Tamas Palfy

> PutParquet, PutORC processors may hang after writing to ADLS
> 
>
> Key: NIFI-9436
> URL: https://issues.apache.org/jira/browse/NIFI-9436
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
>
> h2. Background
> In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
> *org.apache.parquet.hadoop.ParquetWriter* writer is _created_, used to write 
> record to an HDFS location and is later _closed_ explicitly.
> The writer creation process involves the instantiation of an 
> *org.apache.hadoop.fs.FileSystem* object, which, when writing to ADLS is 
> going to be an *org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem*.
> Note that the NiFi AbstractPutHDFSRecord processor already created a 
> FileSystem object of the same type for it's own purposes but the writer 
> creates it's own.
> The writer only uses the FileSystem object to create an 
> *org.apache.hadoop.fs.azurebfs.servicesAbfsOutputStream* object and doesn't 
> keep the FileSystem object itself.
> This makes the FileSystem object eligible for garbage collection.
> The AbfsOutputStream writes data asynchronously. Submits the task to an 
> executorservice and stores it in a collection.
> h2. The issue
> * The _AzureBlobFileSystem_ and the _AbfsOutputStream_ both have reference to 
> the same _ThreadPoolExecutor_ object.
> * _AzureBlobFileSystem_ (probably depending on the version) overrides the 
> _finalize()_ method and closes itself when that is called. This involves 
> shutting down the referenced _ThreadPoolExecutor_.
> * It's possible for garbage collection to occur after the _ParquetWriter_ is 
> created but before explicitly closing it. GC -> 
> _AzureBlobFileSystem.finalize()_ -> _ThreadPoolExecutor.shutdown()_.
> * When the _ParquetWriter_ is explicitly closed it tries to run a cleanup job 
> using the _ThreadPoolExecutor_. That job submission fails as the 
> _ThreadPoolExecutor_ is already terminated but a _Future_ object is still 
> created -  and is being wait for indefinitely.
> This causes the processor to hang.
> h2. The solution
> This feels like an issue that should be addressed in the _hadoop-azure_ 
> library but it's possible to apply a workaround in NiFi.
> The problem starts by the _AzureBlobFileSystem_ getting garbage collected. So 
> if the _ParquetWriter_ used the same _FileSystem_ object that the processor 
> already created for itself -and kept the reference for- it would prevent the 
> garbage collection to occur.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (NIFI-9433) Load Balancer hangs

2021-12-02 Thread Mark Bean (Jira)



 [ 
https://issues.apache.org/jira/browse/NIFI-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Bean updated NIFI-9433:

Description: 
Simplified scenario to demonstrate problem:
A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
connection -> UpdateAttribute. And, unconnected to the first two processors, 
Funnel #1 -> non-load-balanced Connection -> Funnel #2.
GenerateFlowFile is scheduled to run on Primary Node only. It is started. This 
causes the connection to be very busy load balancing (round robin). Then, the 
connection between the two funnels is removed.
Immediately, an error is thrown, and the flow gets stuck in a state of 
constantly throwing errors indicating that a connection (the one just deleted) 
does not exist and cannot be balanced.
It is unclear why this connection is being considered by the load balancer at 
all.

The sequence of errors include the following:
Primary Node reports 
2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged from 
FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
Unacknowledged=[-206, -20600 Bytes] ]
java.lang.RuntimeException: Cannot create negative queue size
2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from FlowFile 
Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ ActiveQueue=[206, 
20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], Unacknowledged=[-206, 
-20600 Bytes] ]
java.lang.RuntimeException: Cannot create negative queue size

The above may be a symptom of subsequent errors in the log:
Primary Node reports:
2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 

java.io.IOException: Failed to negotiate Protocol Version with Peer 
. Recommended version 1 but instead of an ACCEPT or REJECT response 
got back a response of 33.

Non-Primary Node reports:
2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with 
Peer
java.io.IOException: Expected to receive Transaction Completion Indicator from 
Peer  but instead received a value of 1

The highly concerning part is this error which indicates a Connection which was 
not scheduled to load balance was attempting to receive a FlowFile.
Non-Primary Node reports:
2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
Peer  for Connection with ID  but no connection exists with that ID.

Note the that  value in this message corresponds to the Connection that 
was removed causing the errors to begin. Should the above message ever occur? 
Does the load balancer ever consider Connections which are configured as "Do 
not load balance"

Users have also reported that FlowFiles have been load balanced from one 
Connection to another, unrelated Connection on the other Node. (This is still 
being verified.)

Finally, on the UI the load-balanced connection indicates it is actively load 
balancing some number (206 in this case) of FlowFiles currently in the 
connection. And, attempts to "list queue" on this connection show no FlowFiles. 
Presumably they are being held by the load balancer and are inaccessible in the 
queue.

  was:
Simplified scenario to demonstrate problem:
A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
connection -> UpdateAttribute. And, unconnected to the first two processors, 
Funnel #1 -> non-load-balanced Connection -> Funnel #2.
GenerateFlowFile is scheduled to run on Primary Node only. It is started. This 
causes the connection to be very busy load balancing (round robin). Then, the 
connection between the two funnels is removed.
Immediately, an error is thrown, and the flow gets stuck in a state of 
constantly throwing errors indicating that a connection (the one just deleted) 
does not exist and cannot be balanced.
It is unclear why this connection is being considered by the load balancer at 
all.

The sequence of errors include the following:
Primary Node reports 
2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged from 
FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
Unacknowledged=[-206, -20600 Bytes] ]
java.lang.RuntimeException: Cannot create negative queue size
2021-12-02 12:20:03,813 ERROR [NiFi Web

[jira] [Created] (NIFI-9436) PutParquet, PutORC processors may hang after writing to ADLS

Tamas Palfy created NIFI-9436:
-

 Summary: PutParquet, PutORC processors may hang after writing to 
ADLS
 Key: NIFI-9436
 URL: https://issues.apache.org/jira/browse/NIFI-9436
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Tamas Palfy


h2. Background
In *AbstractPutHDFSRecord* (of which *PutParquet* and *PutORC* is derived) an 
*org.apache.parquet.hadoop.ParquetWriter* writer is _created_, used to write 
record to an HDFS location and is later _closed_ explicitly.

The writer creation process involves the instantiation of an 
org.apache.hadoop.fs.FileSystem object, which, when writing to ADLS is going to 
be an org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.
Note that the NiFi AbstractPutHDFSRecord processor already created a FileSystem 
object of the same type for it's own purposes but the writer creates it's own.

The writer only uses the FileSystem object to create an AbfsOutputStream object 
and doesn't keep the FileSystem object itself.
This makes the FileSystem object eligible to garbage collection.

The AbfsOutputStream writes data asynchronously. Submits the task to an 
executorservice and stores it in a collection.

h2. The issue

* The AzureBlobFileSystem and the AbfsOutputStream both have reference to the 
same ThreadPoolExecutor object.
* AzureBlobFileSystem -probably depending on the version- overrides the 
finalize() method and closes itself when that is called. This involves shutting 
down the referenced ThreadPoolExecutor.
* It's possible for garbage collection to occur after the ParquetWriter is 
created but before explicitly closing it. GC -> AzureBlobFileSystem.finalize() 
-> ThreadPoolExecutor.shutdown().
* When the ParquetWriter is explicitly closed it tries to run a cleanup job 
using the ThreadPoolExecutor. That job submission fails as the 
ThreadPoolExecutor is already terminated but a Future object is still created - 
 and is being wait for indefinitely.

This causes the processor to hang.

h2. The solution

This feels like an issue that should be addressed in the hadoop-azure library 
but it's possible to apply a workaround in NiFi.

The problem starts by the AzureBlobFileSystem getting garbage collected. So if 
the ParquetWriter used the same FileSystem object that the processor already 
created for itself -and kept the reference for- it would prevent the garbage 
collection to occur.





--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-9435) Add Filtering Parameters to Flow Metrics

David Handermann created NIFI-9435:
--

 Summary: Add Filtering Parameters to Flow Metrics
 Key: NIFI-9435
 URL: https://issues.apache.org/jira/browse/NIFI-9435
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: David Handermann
Assignee: David Handermann


The NiFi REST API includes support for generating metrics using Prometheus as 
the specified Producer. The current implementation gathers metrics in multiple 
registries, including Process Groups, Connections, Bulletins, and Java Virtual 
Machine status.

On installations with a large number of components, the metrics resource can 
generate a very large amount of information, resulting in timeouts when 
requesting clients have low timeout settings. In some cases, returning all 
metrics may not be necessary. Adding optional filter parameters to the REST API 
would support returning a smaller amount of information, providing better 
support for NiFi installations with a large number of components. Potential 
filtering parameters could include limiting the selected registry or metrics 
component type.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-9433) Load Balancer hangs

2021-12-02 Thread Mark Bean (Jira)

Mark Bean created NIFI-9433:
---

 Summary: Load Balancer hangs
 Key: NIFI-9433
 URL: https://issues.apache.org/jira/browse/NIFI-9433
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.15.0
Reporter: Mark Bean


Simplified scenario to demonstrate problem:
A 2-node cluster with a simple flow. GenerateFlowFile -> load-balanced 
connection -> UpdateAttribute. And, unconnected to the first two processors, 
Funnel #1 -> non-load-balanced Connection -> Funnel #2.
GenerateFlowFile is scheduled to run on Primary Node only. It is started. This 
causes the connection to be very busy load balancing (round robin). Then, the 
connection between the two funnels is removed.
Immediately, an error is thrown, and the flow gets stuck in a state of 
constantly throwing errors indicating that a connection (the one just deleted) 
does not exist and cannot be balanced.
It is unclear why this connection is being considered by the load balancer at 
all.

The sequence of errors include the following:
Primary Node reports 
2021-12-02 12:20:03,812 ERROR [NiFi Web Server-1811] 
o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged from 
FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap 
Files=[0], Unacknowledged=[0, 0 Bytes] ] to FlowFile Queue Size[ 
ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
Unacknowledged=[-206, -20600 Bytes] ]
java.lang.RuntimeException: Cannot create negative queue size
2021-12-02 12:20:03,813 ERROR [NiFi Web Server-1811] 
o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue active from FlowFile 
Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], 
Unacknowledged=[-206, -20600 Bytes] ] to FlowFile Queue Size[ ActiveQueue=[206, 
20600 Bytes], Swap Queue=[0, 0 Bytes], Swap Files=[0], Unacknowledged=[-206, 
-20600 Bytes] ]
java.lang.RuntimeException: Cannot create negative queue size

The above may be a symptom of subsequent errors in the log:
Primary Node reports:
2021-12-02 12:20:03,814 ERROR [Load-Balanced Client Thread-6] 
o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer 
{host:port}
java.io.IOException: Failed to negotiate Protocol Version with Peer 
{host:port}. Recommended version 1 but instead of an ACCEPT or REJECT response 
got back a response of 33.

Non-Primary Node reports:
2021-12-02 12:20:03,828 ERROR [Load-Balance Server Thread-4] 
o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer 
{fqdn/IP:port}
java.io.IOException: Expected to receive Transaction Completion Indicator from 
Peer {fqdn} but instead received a value of 1

The highly concerning part is this error which indicates a Connection which was 
not scheduled to load balance was attempting to receive a FlowFile.
Non-Primary Node reports:
2021-12-02 12:29:05,228 ERROR [Load-Balance Server Thread-808] 
o.a.n.c.q.c.s.StandardLoadBalanceProtocol Attempted to receive FlowFiles from 
Peer {fqdn} for Connection with ID {uuid} but no connection exists with that ID.

Note the that {uuid} value in this message corresponds to the Connection that 
was removed causing the errors to begin. Should the above message ever occur? 
Does the load balancer ever consider Connections which are configured as "Do 
not load balance"

Users have also reported that FlowFiles have been load balanced from one 
Connection to another, unrelated Connection on the other Node. (This is still 
being verified.)

Finally, on the UI the load-balanced connection indicates it is actively load 
balancing some number (206 in this case) of FlowFiles currently in the 
connection. And, attempts to "list queue" on this connection show no FlowFiles. 
Presumably they are being held by the load balancer and are inaccessible in the 
queue.





--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-9434) Controller level bulletin when content repository backpressure

Lehel Boér created NIFI-9434:


 Summary: Controller level bulletin when content repository 
backpressure
 Key: NIFI-9434
 URL: https://issues.apache.org/jira/browse/NIFI-9434
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Lehel Boér


We see more and more users hitting the situation where NiFi seems stuck because 
of the content repo back pressure mechanism that is set by default. We should 
have a controller level bulletin to let the users know what is happening.

This usually happens when all repositories and OS data are using the same file 
system and usage goes over 52%.

If nifi.content.repository.archive.max.usage.percentage is 50% (default value) 
and nifi.content.repository.archive.backpressure.percentage is not set, the 
effective value of nifi.content.repository.archive.backpressure.percentage will 
be 52%.

This property is used to control the content repository disk usage percentage 
at which backpressure is applied to the processes writing to the content 
repository. Once this percentage is reached, the content repository will refuse 
any additional writes. Writes will be refused until the archive delete process 
has brought the content repository disk usage percentage below 
nifi.content.repository.archive.max.usage.percentage.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1222: MINIFICPP-1695 Fix execution of native python processors



lordgamez opened a new pull request #1222:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1222


   PR https://github.com/apache/nifi-minifi-cpp/pull/1126 fixed the execution 
of ExecutePythonProcessor processor, but unfortunately broke the execution of 
native python processors. The ExecutePythonProcessor implements a solution now 
that can host the native python processors but can also be run individually 
with a script property parameter.
   
   

   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically main)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI 
results for build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] pvillard31 opened a new pull request #5562: NIFI-9432 - fix typo in diagnostic output



pvillard31 opened a new pull request #5562:
URL: https://github.com/apache/nifi/pull/5562


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Enables X functionality; fixes bug NIFI-._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `main`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (NIFI-9432) Typo in Diagnostic data - Input/Output Ports

2021-12-02 Thread Pierre Villard (Jira)



 [ 
https://issues.apache.org/jira/browse/NIFI-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-9432:
-
Status: Patch Available  (was: Open)

> Typo in Diagnostic data - Input/Output Ports
> 
>
> Key: NIFI-9432
> URL: https://issues.apache.org/jira/browse/NIFI-9432
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> In the diagnostic data, we see "Input" twice when it should be Input and 
> Output:
> {code:java}
> Site-to-Site Input Ports: 0
> Site-to-Site Input Ports: 0{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-9432) Typo in Diagnostic data - Input/Output Ports

2021-12-02 Thread Pierre Villard (Jira)

Pierre Villard created NIFI-9432:


 Summary: Typo in Diagnostic data - Input/Output Ports
 Key: NIFI-9432
 URL: https://issues.apache.org/jira/browse/NIFI-9432
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Pierre Villard
Assignee: Pierre Villard


In the diagnostic data, we see "Input" twice when it should be Input and Output:
{code:java}
Site-to-Site Input Ports: 0
Site-to-Site Input Ports: 0{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MINIFICPP-1695) Fix execution of native python processors



 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gábor Gyimesi updated MINIFICPP-1695:
-
Description: Make sure native python processors and also 
ExecutePythonProcessor can be individually executed.  (was: There are several 
native python processors under the extensions/pythonprocessors that do not have 
any tests. We should provide test coverage for these processors.)

> Fix execution of native python processors
> -
>
> Key: MINIFICPP-1695
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1695
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gábor Gyimesi
>Assignee: Gábor Gyimesi
>Priority: Minor
>
> Make sure native python processors and also ExecutePythonProcessor can be 
> individually executed.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MINIFICPP-1695) Fix execution of native python processors



 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gábor Gyimesi updated MINIFICPP-1695:
-
Issue Type: Bug  (was: Improvement)

> Fix execution of native python processors
> -
>
> Key: MINIFICPP-1695
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1695
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gábor Gyimesi
>Assignee: Gábor Gyimesi
>Priority: Minor
>
> Make sure native python processors and also ExecutePythonProcessor can be 
> individually executed.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (NIFI-8706) Add support for Redis Lists and Hashes

2021-12-02 Thread Otto Fowler (Jira)



[ 
https://issues.apache.org/jira/browse/NIFI-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452393#comment-17452393
 ] 

Otto Fowler commented on NIFI-8706:
---

These are great examples of redis commands, can you put them in a nifi context?

"I would like to write attributes to redis"?
"I would like to write one or more fields from records to redis"
"I would like to write fields from records that match a record query to redis"
"I would like have a record writer that writes json to redis"

something like that?

> Add support for Redis Lists and Hashes
> --
>
> Key: NIFI-8706
> URL: https://issues.apache.org/jira/browse/NIFI-8706
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.8.0, 1.13.2
>Reporter: Doug Houck
>Priority: Major
>  Labels: newbie, performance
>
> Nifi supports Redis interactions, but only for Keys.  From a Redis 
> perspective, this is a poor use of resources.  Lists and Hashes only required 
> one key per.  It would be nice to have the ability to interact with Redis 
> Lists and Hashes in a Nifi Processor.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MINIFICPP-1695) Fix execution of native python processors



 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gábor Gyimesi updated MINIFICPP-1695:
-
Summary: Fix execution of native python processors  (was: Add test coverage 
for python processors)

> Fix execution of native python processors
> -
>
> Key: MINIFICPP-1695
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1695
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gábor Gyimesi
>Assignee: Gábor Gyimesi
>Priority: Minor
>
> There are several native python processors under the 
> extensions/pythonprocessors that do not have any tests. We should provide 
> test coverage for these processors.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-9431) Issue While Migrating to 1.14.0

2021-12-02 Thread Sarah Alkhudair (Jira)

Sarah Alkhudair created NIFI-9431:
-

 Summary: Issue While Migrating to 1.14.0 
 Key: NIFI-9431
 URL: https://issues.apache.org/jira/browse/NIFI-9431
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Sarah Alkhudair


Dears,

We are facing an issue in migrating from 1.13.2 to 1.14.0. When we upload 
templates (.xml files) to 1.14.0 and then restart NiFi ,we keep having this 
error back (WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup 
of context o.e.j.w.WebAppContext).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors



lordgamez commented on a change in pull request #1216:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1216#discussion_r760940317



##
File path: docker/test/integration/features/opcua.feature
##
@@ -0,0 +1,153 @@
+Feature: Putting and fetching data to OPC UA server
+  In order to send and fetch data from an OPC UA server
+  As a user of MiNiFi
+  I need to have PutOPCProcessor and FetchOPCProcessor
+
+  Background:
+Given the content of "/tmp/output" is monitored
+
+  Scenario Outline: Create and fetch data from an OPC UA node
+Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input" in the "create-opc-ua-node" flow
+And a file with the content "" is present in "/tmp/input"
+And a PutOPCProcessor processor in the "create-opc-ua-node" flow
+And a FetchOPCProcessor processor in the "fetch-opc-ua-node" flow
+And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "fetch-opc-ua-node" flow
+And these processor properties are set:
+  | processor name| property name   | property value   
   |
+  | PutOPCProcessor   | Parent node ID  | 85   
   |
+  | PutOPCProcessor   | Parent node ID type | Int  
   |
+  | PutOPCProcessor   | Target node ID  |  
   |
+  | PutOPCProcessor   | Target node ID type | Int  
   |
+  | PutOPCProcessor   | Target node namespace index | 1
   |
+  | PutOPCProcessor   | Value type  |  
   |
+  | PutOPCProcessor   | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | PutOPCProcessor   | Target node browse name | testnodename 
   |
+  | FetchOPCProcessor | Node ID |  
   |
+  | FetchOPCProcessor | Node ID type| Int  
   |
+  | FetchOPCProcessor | Namespace index | 1
   |
+  | FetchOPCProcessor | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | FetchOPCProcessor | Max depth   | 1
   |
+
+And the "success" relationship of the GetFile processor is connected to 
the PutOPCProcessor
+And the "success" relationship of the FetchOPCProcessor processor is 
connected to the PutFile
+
+And an OPC UA server is set up
+
+When all instances start up
+Then at least one flowfile with the content "" is placed in the 
monitored directory in less than 60 seconds
+
+  Examples: Topic names and formats to test
+| Value Type   | Value   |
+| String   | Test|
+| UInt32   | 42  |
+| Double   | 123.321 |
+| Boolean  | False   |
+
+  Scenario Outline: Update and fetch data from an OPC UA node

Review comment:
   Sure, added a comment in 1663b821d5a73afb19902a47a8fc7620d9f7a380




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors



fgerlits commented on a change in pull request #1216:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1216#discussion_r760925755



##
File path: docker/test/integration/features/opcua.feature
##
@@ -0,0 +1,153 @@
+Feature: Putting and fetching data to OPC UA server
+  In order to send and fetch data from an OPC UA server
+  As a user of MiNiFi
+  I need to have PutOPCProcessor and FetchOPCProcessor
+
+  Background:
+Given the content of "/tmp/output" is monitored
+
+  Scenario Outline: Create and fetch data from an OPC UA node
+Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input" in the "create-opc-ua-node" flow
+And a file with the content "" is present in "/tmp/input"
+And a PutOPCProcessor processor in the "create-opc-ua-node" flow
+And a FetchOPCProcessor processor in the "fetch-opc-ua-node" flow
+And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "fetch-opc-ua-node" flow
+And these processor properties are set:
+  | processor name| property name   | property value   
   |
+  | PutOPCProcessor   | Parent node ID  | 85   
   |
+  | PutOPCProcessor   | Parent node ID type | Int  
   |
+  | PutOPCProcessor   | Target node ID  |  
   |
+  | PutOPCProcessor   | Target node ID type | Int  
   |
+  | PutOPCProcessor   | Target node namespace index | 1
   |
+  | PutOPCProcessor   | Value type  |  
   |
+  | PutOPCProcessor   | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | PutOPCProcessor   | Target node browse name | testnodename 
   |
+  | FetchOPCProcessor | Node ID |  
   |
+  | FetchOPCProcessor | Node ID type| Int  
   |
+  | FetchOPCProcessor | Namespace index | 1
   |
+  | FetchOPCProcessor | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | FetchOPCProcessor | Max depth   | 1
   |
+
+And the "success" relationship of the GetFile processor is connected to 
the PutOPCProcessor
+And the "success" relationship of the FetchOPCProcessor processor is 
connected to the PutFile
+
+And an OPC UA server is set up
+
+When all instances start up
+Then at least one flowfile with the content "" is placed in the 
monitored directory in less than 60 seconds
+
+  Examples: Topic names and formats to test
+| Value Type   | Value   |
+| String   | Test|
+| UInt32   | 42  |
+| Double   | 123.321 |
+| Boolean  | False   |
+
+  Scenario Outline: Update and fetch data from an OPC UA node

Review comment:
   Can you add a comment about this?  Currently this is not visible 
anywhere in the code changes of the PR.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors



lordgamez commented on a change in pull request #1216:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1216#discussion_r760886428



##
File path: docker/test/integration/MiNiFi_integration_test_driver.py
##
@@ -141,7 +141,7 @@ def check_for_multiple_files_generated(self, file_count, 
timeout_seconds, expect
 def check_for_at_least_one_file_with_content_generated(self, content, 
timeout_seconds):
 output_validator = 
SingleOrMultiFileOutputValidator(decode_escaped_str(content))
 
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
-self.check_output(timeout_seconds, output_validator, 1)
+self.check_output(timeout_seconds, output_validator, timeout_seconds)

Review comment:
   I agree, that looks better, updated in 
5ce4c54f1f26ab113d81b813c7f1a4370f7cc9a6




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors



lordgamez commented on a change in pull request #1216:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1216#discussion_r760884568



##
File path: docker/test/integration/features/opcua.feature
##
@@ -0,0 +1,153 @@
+Feature: Putting and fetching data to OPC UA server
+  In order to send and fetch data from an OPC UA server
+  As a user of MiNiFi
+  I need to have PutOPCProcessor and FetchOPCProcessor
+
+  Background:
+Given the content of "/tmp/output" is monitored
+
+  Scenario Outline: Create and fetch data from an OPC UA node
+Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input" in the "create-opc-ua-node" flow
+And a file with the content "" is present in "/tmp/input"
+And a PutOPCProcessor processor in the "create-opc-ua-node" flow
+And a FetchOPCProcessor processor in the "fetch-opc-ua-node" flow
+And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "fetch-opc-ua-node" flow
+And these processor properties are set:
+  | processor name| property name   | property value   
   |
+  | PutOPCProcessor   | Parent node ID  | 85   
   |
+  | PutOPCProcessor   | Parent node ID type | Int  
   |
+  | PutOPCProcessor   | Target node ID  |  
   |
+  | PutOPCProcessor   | Target node ID type | Int  
   |
+  | PutOPCProcessor   | Target node namespace index | 1
   |
+  | PutOPCProcessor   | Value type  |  
   |
+  | PutOPCProcessor   | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | PutOPCProcessor   | Target node browse name | testnodename 
   |
+  | FetchOPCProcessor | Node ID |  
   |
+  | FetchOPCProcessor | Node ID type| Int  
   |
+  | FetchOPCProcessor | Namespace index | 1
   |
+  | FetchOPCProcessor | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | FetchOPCProcessor | Max depth   | 1
   |
+
+And the "success" relationship of the GetFile processor is connected to 
the PutOPCProcessor
+And the "success" relationship of the FetchOPCProcessor processor is 
connected to the PutFile
+
+And an OPC UA server is set up
+
+When all instances start up
+Then at least one flowfile with the content "" is placed in the 
monitored directory in less than 60 seconds
+
+  Examples: Topic names and formats to test
+| Value Type   | Value   |
+| String   | Test|
+| UInt32   | 42  |
+| Double   | 123.321 |
+| Boolean  | False   |
+
+  Scenario Outline: Update and fetch data from an OPC UA node

Review comment:
   Yes, the test server application which is started from the container 
image already defines a list of demo values for each node type of OPC UA.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors



fgerlits commented on a change in pull request #1216:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1216#discussion_r760876956



##
File path: docker/test/integration/features/opcua.feature
##
@@ -0,0 +1,153 @@
+Feature: Putting and fetching data to OPC UA server
+  In order to send and fetch data from an OPC UA server
+  As a user of MiNiFi
+  I need to have PutOPCProcessor and FetchOPCProcessor
+
+  Background:
+Given the content of "/tmp/output" is monitored
+
+  Scenario Outline: Create and fetch data from an OPC UA node
+Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input" in the "create-opc-ua-node" flow
+And a file with the content "" is present in "/tmp/input"
+And a PutOPCProcessor processor in the "create-opc-ua-node" flow
+And a FetchOPCProcessor processor in the "fetch-opc-ua-node" flow
+And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "fetch-opc-ua-node" flow
+And these processor properties are set:
+  | processor name| property name   | property value   
   |
+  | PutOPCProcessor   | Parent node ID  | 85   
   |
+  | PutOPCProcessor   | Parent node ID type | Int  
   |
+  | PutOPCProcessor   | Target node ID  |  
   |
+  | PutOPCProcessor   | Target node ID type | Int  
   |
+  | PutOPCProcessor   | Target node namespace index | 1
   |
+  | PutOPCProcessor   | Value type  |  
   |
+  | PutOPCProcessor   | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | PutOPCProcessor   | Target node browse name | testnodename 
   |
+  | FetchOPCProcessor | Node ID |  
   |
+  | FetchOPCProcessor | Node ID type| Int  
   |
+  | FetchOPCProcessor | Namespace index | 1
   |
+  | FetchOPCProcessor | OPC server endpoint | 
opc.tcp://opcua-server:4840/|
+  | FetchOPCProcessor | Max depth   | 1
   |
+
+And the "success" relationship of the GetFile processor is connected to 
the PutOPCProcessor
+And the "success" relationship of the FetchOPCProcessor processor is 
connected to the PutFile
+
+And an OPC UA server is set up
+
+When all instances start up
+Then at least one flowfile with the content "" is placed in the 
monitored directory in less than 60 seconds
+
+  Examples: Topic names and formats to test
+| Value Type   | Value   |
+| String   | Test|
+| UInt32   | 42  |
+| Double   | 123.321 |
+| Boolean  | False   |
+
+  Scenario Outline: Update and fetch data from an OPC UA node

Review comment:
   Sorry, I don't understand -- are the node IDs 51034, 51001 etc defined 
in the `lordgamez/open62541` image?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1221: MINIFICPP-1630 Create FetchAzureDataLakeStorage processor



lordgamez opened a new pull request #1221:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1221


   https://issues.apache.org/jira/browse/MINIFICPP-1630
   
   
--
   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically main)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI 
results for build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1216: MINIFICPP-1290 Create test coverage for OPC processors