[jira] [Commented] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775121#comment-17775121 ] ASF subversion and git services commented on NIFI-12122: Commit 61c856a54c56a1180a621bc8f17f0ce63044fcc8 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=61c856a54c ] NIFI-12122 Corrected test for Parameter Context updating Signed-off-by: David Handermann > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Assignee: Zackeus Bengtsson >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775119#comment-17775119 ] ASF subversion and git services commented on NIFI-12122: Commit 1cb7614b0b5fe02ed883e3fcfaf7212aa5f20984 in nifi's branch refs/heads/support/nifi-1.x from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1cb7614b0b ] NIFI-12122 Corrected TestFlowController references - Adjusted XmlFlowSynchronizer method invocation for backported changes Signed-off-by: David Handermann > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Assignee: Zackeus Bengtsson >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12122: --- Assignee: Zackeus Bengtsson > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Assignee: Zackeus Bengtsson >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12160) Check if all the necessary nars have been fully unpacked before starting Kafka Connector
[ https://issues.apache.org/jira/browse/NIFI-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12160. - Fix Version/s: 2.0.0 1.24.0 Resolution: Fixed > Check if all the necessary nars have been fully unpacked before starting > Kafka Connector > > > Key: NIFI-12160 > URL: https://issues.apache.org/jira/browse/NIFI-12160 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Labels: kafka-connect > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Each deployed Stateless NiFi-based Kafka Connect connector has a directory > under DEFAULT_WORKING_DIRECTORY (/tmp/nifi-stateless-working) where the > necessary NARs are unpacked. > If the job doing the unpacking is interrupted for some reason, the > connector's directory will exist without having all the necessary files in > it. In a case like this when the connector is started it detects the > directory so it does not unpack the NARs again however the missing files > cause various errors. > Currently the solution is manual: the connector's directory needs to be > deleted from /tmp/nifi-stateless-working and the connector needs to be > redeployed. > A fix should be created for this issue so that the system detects that > unpacking hadn't been finished and does the unpacking properly before > deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12122. - Fix Version/s: 2.0.0 1.24.0 Resolution: Fixed > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775094#comment-17775094 ] ASF subversion and git services commented on NIFI-12122: Commit 387c22b22c8583082d15acd64868c0e3bd1454e3 in nifi's branch refs/heads/support/nifi-1.x from Zackeus Bengtsson [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=387c22b22c ] NIFI-12122 Fixed persistence of Parameter Context descriptions Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty This closes: #7787 Signed-off-by: David Handermann (cherry picked from commit 0a47157640c1502a0379a7f66d764d2d7d344872) > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12160) Check if all the necessary nars have been fully unpacked before starting a connector
[ https://issues.apache.org/jira/browse/NIFI-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775095#comment-17775095 ] ASF subversion and git services commented on NIFI-12160: Commit 19153a7609f7c1a4a229d3cf9186903f1b2da86a in nifi's branch refs/heads/support/nifi-1.x from Peter Gyori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=19153a7609 ] NIFI-12160 Kafka Connect: Check for NAR unpacking before starting Check that required NAR files are unpacked completely before starting the Kafka Connector This closes #7832 Signed-off-by: David Handermann (cherry picked from commit b2e3898e171b89e25b5e06f6ee849bb3edabbf9a) > Check if all the necessary nars have been fully unpacked before starting a > connector > > > Key: NIFI-12160 > URL: https://issues.apache.org/jira/browse/NIFI-12160 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Labels: kafka-connect > Time Spent: 1h 10m > Remaining Estimate: 0h > > Each deployed Stateless NiFi-based Kafka Connect connector has a directory > under DEFAULT_WORKING_DIRECTORY (/tmp/nifi-stateless-working) where the > necessary NARs are unpacked. > If the job doing the unpacking is interrupted for some reason, the > connector's directory will exist without having all the necessary files in > it. In a case like this when the connector is started it detects the > directory so it does not unpack the NARs again however the missing files > cause various errors. > Currently the solution is manual: the connector's directory needs to be > deleted from /tmp/nifi-stateless-working and the connector needs to be > redeployed. > A fix should be created for this issue so that the system detects that > unpacking hadn't been finished and does the unpacking properly before > deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12160) Check if all the necessary nars have been fully unpacked before starting Kafka Connector
[ https://issues.apache.org/jira/browse/NIFI-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12160: Summary: Check if all the necessary nars have been fully unpacked before starting Kafka Connector (was: Check if all the necessary nars have been fully unpacked before starting a connector) > Check if all the necessary nars have been fully unpacked before starting > Kafka Connector > > > Key: NIFI-12160 > URL: https://issues.apache.org/jira/browse/NIFI-12160 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Labels: kafka-connect > Time Spent: 1h 10m > Remaining Estimate: 0h > > Each deployed Stateless NiFi-based Kafka Connect connector has a directory > under DEFAULT_WORKING_DIRECTORY (/tmp/nifi-stateless-working) where the > necessary NARs are unpacked. > If the job doing the unpacking is interrupted for some reason, the > connector's directory will exist without having all the necessary files in > it. In a case like this when the connector is started it detects the > directory so it does not unpack the NARs again however the missing files > cause various errors. > Currently the solution is manual: the connector's directory needs to be > deleted from /tmp/nifi-stateless-working and the connector needs to be > redeployed. > A fix should be created for this issue so that the system detects that > unpacking hadn't been finished and does the unpacking properly before > deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12160) Check if all the necessary nars have been fully unpacked before starting a connector
[ https://issues.apache.org/jira/browse/NIFI-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775093#comment-17775093 ] ASF subversion and git services commented on NIFI-12160: Commit b2e3898e171b89e25b5e06f6ee849bb3edabbf9a in nifi's branch refs/heads/main from Peter Gyori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b2e3898e17 ] NIFI-12160 Kafka Connect: Check for NAR unpacking before starting Check that required NAR files are unpacked completely before starting the Kafka Connector This closes #7832 Signed-off-by: David Handermann > Check if all the necessary nars have been fully unpacked before starting a > connector > > > Key: NIFI-12160 > URL: https://issues.apache.org/jira/browse/NIFI-12160 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Labels: kafka-connect > Time Spent: 1h 10m > Remaining Estimate: 0h > > Each deployed Stateless NiFi-based Kafka Connect connector has a directory > under DEFAULT_WORKING_DIRECTORY (/tmp/nifi-stateless-working) where the > necessary NARs are unpacked. > If the job doing the unpacking is interrupted for some reason, the > connector's directory will exist without having all the necessary files in > it. In a case like this when the connector is started it detects the > directory so it does not unpack the NARs again however the missing files > cause various errors. > Currently the solution is manual: the connector's directory needs to be > deleted from /tmp/nifi-stateless-working and the connector needs to be > redeployed. > A fix should be created for this issue so that the system detects that > unpacking hadn't been finished and does the unpacking properly before > deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12122) Parameter Context description gets lost on NiFi restart
[ https://issues.apache.org/jira/browse/NIFI-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775092#comment-17775092 ] ASF subversion and git services commented on NIFI-12122: Commit 0a47157640c1502a0379a7f66d764d2d7d344872 in nifi's branch refs/heads/main from Zackeus Bengtsson [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0a47157640 ] NIFI-12122 Fixed persistence of Parameter Context descriptions Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty This closes: #7787 Signed-off-by: David Handermann > Parameter Context description gets lost on NiFi restart > --- > > Key: NIFI-12122 > URL: https://issues.apache.org/jira/browse/NIFI-12122 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, > 1.18.0, 1.19.0, 1.20.0, 1.22.0, 1.23.0 >Reporter: Zackeus Bengtsson >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > When creating a parameter context and adding a description it gets persisted > to the flow.json.gz flow definition. If NiFi is shut down, and then > restarted, the parameter context is loaded by the controller and FlowManager, > but the description is not loaded. The flow is then saved to disk, loosing > the parameter context description in the process. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12160: Kafka Connect: Check if all the necessary nars have been … [nifi]
exceptionfactory closed pull request #7832: NIFI-12160: Kafka Connect: Check if all the necessary nars have been … URL: https://github.com/apache/nifi/pull/7832 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12122 Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty [nifi]
exceptionfactory closed pull request #7787: NIFI-12122 Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty URL: https://github.com/apache/nifi/pull/7787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11197 Initial check in for Yaml record reader [nifi]
exceptionfactory commented on PR #7665: URL: https://github.com/apache/nifi/pull/7665#issuecomment-1762354391 > @exceptionfactory I am trying to rework my code to handle the changes made in #7823 for NIFI-12153. What I am discovering is that YAML supports comments and there is no way to turn off parsing of comments. In addition I am trying to mimic `TestJsonTreeRowRecordReader#testReadJSONStringTooLong` and I am trying to use `StreamReadConstraints.builder().maxStringLength(1).build()` yet no exception is thrown even though I have strings longer than one character long. It would seem that aspect only works for Json parsing and not Yaml parsing. I did find the following article [How to parse large YAML file in Java or Kotlin](https://stackoverflow.com/questions/74805240/how-to-parse-large-yaml-file-in-java-or-kotlin) which would seem to handle larger Yaml files but I am not sure if that is the same setting as `StreamReadConstraints.builder().maxStringLength(1).build()`. Please advise. Thanks for the details @dan-s1. If the YAML implementation does not support the same constraints, then I recommend not including those property descriptors on the YAML Record Reader implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-7194) Record Path (and Readers) should be able to select/filter on fields within arrays
[
https://issues.apache.org/jira/browse/NIFI-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775068#comment-17775068
]
Chris Sampson edited comment on NIFI-7194 at 10/13/23 9:19 PM:
---
Partially resolved by NIFI-12124
was (Author: chris s):
Partially resolved by NIFI-12194
> Record Path (and Readers) should be able to select/filter on fields within
> arrays
> -
>
> Key: NIFI-7194
> URL: https://issues.apache.org/jira/browse/NIFI-7194
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.11.1
>Reporter: Chris Sampson
>Priority: Minor
>
> Given a sample input like
> {code:json}
> {
> "field": "value",
> "a_date": "2020-02-01",
> "ids": [
> {
> "id": "1",
> "id_date": "2020-02-24"
> },
> {
> "id": "2",
> "id_date": "2020-02-23"
> }
> ],
> "another": {
> "some_date": "2020-02-02"
> }
> }
> {code}
> It would be useful if one could use NiFi's Record Path to generically
> reference fields nested within arrays as well as top-level/nested objects in
> order to make updates to their values, e.g. as part of a call to the
> UpdateRecord processor.
> For example, to reference all non-array fields whose name contains `date`,
> one could currently use: ` //*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`; however there is no equivalent for referencing the
> date fields within the arrays.
> Such a RecordPath could look like `//[0..-1]/*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`, however this is currently marked as invalid by the
> UpdateRecord processor validation (and presumably therefore doesn't work).
> One tricky addition to this would be how to handle multiple levels of nesting
> of arrays/objects and how much descending through the Record tree should be
> possible.
> An addition to this would be to allow for matching of Map fields too, e.g.
> with something like `//[]/*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`.
> An example use-case for this is where incoming data structure is not
> set/known and may contain arrays (or even maps) with date fields where the
> flow is trying to support multiple date formats (e.g. -MM-dd and
> dd/MM/) but wants to harmonise all dates to a single format for further
> processing (e.g. dd/MM/).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-7194) Record Path (and Readers) should be able to select/filter on fields within arrays
[
https://issues.apache.org/jira/browse/NIFI-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775068#comment-17775068
]
Chris Sampson commented on NIFI-7194:
-
Partially resolved by NIFI-12194
> Record Path (and Readers) should be able to select/filter on fields within
> arrays
> -
>
> Key: NIFI-7194
> URL: https://issues.apache.org/jira/browse/NIFI-7194
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.11.1
>Reporter: Chris Sampson
>Priority: Minor
>
> Given a sample input like
> {code:json}
> {
> "field": "value",
> "a_date": "2020-02-01",
> "ids": [
> {
> "id": "1",
> "id_date": "2020-02-24"
> },
> {
> "id": "2",
> "id_date": "2020-02-23"
> }
> ],
> "another": {
> "some_date": "2020-02-02"
> }
> }
> {code}
> It would be useful if one could use NiFi's Record Path to generically
> reference fields nested within arrays as well as top-level/nested objects in
> order to make updates to their values, e.g. as part of a call to the
> UpdateRecord processor.
> For example, to reference all non-array fields whose name contains `date`,
> one could currently use: ` //*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`; however there is no equivalent for referencing the
> date fields within the arrays.
> Such a RecordPath could look like `//[0..-1]/*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`, however this is currently marked as invalid by the
> UpdateRecord processor validation (and presumably therefore doesn't work).
> One tricky addition to this would be how to handle multiple levels of nesting
> of arrays/objects and how much descending through the Record tree should be
> possible.
> An addition to this would be to allow for matching of Map fields too, e.g.
> with something like `//[]/*[matchesRegex(fieldName(.),
> '(^|.+_)date(_.+|$)')]`.
> An example use-case for this is where incoming data structure is not
> set/known and may contain arrays (or even maps) with date fields where the
> flow is trying to support multiple date formats (e.g. -MM-dd and
> dd/MM/) but wants to harmonise all dates to a single format for further
> processing (e.g. dd/MM/).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12124) Add a RenameRecordField processor
[
https://issues.apache.org/jira/browse/NIFI-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775063#comment-17775063
]
ASF subversion and git services commented on NIFI-12124:
Commit 3ae0eedee6ae67c48dd59f68752fa728e4355c41 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3ae0eedee6 ]
NIFI-12124: This closes #7791. Added a new RenameRecordField processor. In
testing, also noticed that the Descendant Wildcard operator (//*) and
Descendant Field Path Operator (//name, for instance) did not properly account
for array of records or map elements, so addressed those concerns.
Signed-off-by: Joseph Witt
> Add a RenameRecordField processor
> -
>
> Key: NIFI-12124
> URL: https://issues.apache.org/jira/browse/NIFI-12124
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The RemoveRecordField processor that was recently added has made it much
> simpler to remove specific fields from a Record. However, it's still not
> trivial to rename a field. There are multiple ways to achieve this, but none
> of them are straight-forward.
> It should take as its configuration user-defined properties such that the key
> is a RecordPath and the value is a String literal or a Expression Language
> Expression - much like UpdateRecord does when configured with "Replacement
> Value Strategy" = "Literal Value." When evaluating EL, it should expose
> FlowFile attributes as well as {{field.name}} and {{{}field.value{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12124) Add a RenameRecordField processor
[
https://issues.apache.org/jira/browse/NIFI-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12124:
Fix Version/s: 2.0.0
(was: 2.latest)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add a RenameRecordField processor
> -
>
> Key: NIFI-12124
> URL: https://issues.apache.org/jira/browse/NIFI-12124
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The RemoveRecordField processor that was recently added has made it much
> simpler to remove specific fields from a Record. However, it's still not
> trivial to rename a field. There are multiple ways to achieve this, but none
> of them are straight-forward.
> It should take as its configuration user-defined properties such that the key
> is a RecordPath and the value is a String literal or a Expression Language
> Expression - much like UpdateRecord does when configured with "Replacement
> Value Strategy" = "Literal Value." When evaluating EL, it should expose
> FlowFile attributes as well as {{field.name}} and {{{}field.value{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12124: Added a new RenameRecordField processor. In testing, also… [nifi]
markap14 commented on PR #7791: URL: https://github.com/apache/nifi/pull/7791#issuecomment-1762219872 Thanks @joewitt -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12124: Added a new RenameRecordField processor. In testing, also… [nifi]
markap14 commented on PR #7791: URL: https://github.com/apache/nifi/pull/7791#issuecomment-1762219803 @ChrisSamo632 the Jira you pointed out appears to get pretty complex. I am not 100% sure if this address all cases identified there, but it will certainly help on that front. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12124: Added a new RenameRecordField processor. In testing, also… [nifi]
joewitt commented on PR #7791: URL: https://github.com/apache/nifi/pull/7791#issuecomment-1762219608 looks good - merged to main. Not sure on nifi7194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12124: Added a new RenameRecordField processor. In testing, also… [nifi]
asfgit closed pull request #7791: NIFI-12124: Added a new RenameRecordField processor. In testing, also… URL: https://github.com/apache/nifi/pull/7791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12221) Make heartbeat responses more lenient in some cases
[ https://issues.apache.org/jira/browse/NIFI-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12221: Fix Version/s: 2.0.0 (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Make heartbeat responses more lenient in some cases > --- > > Key: NIFI-12221 > URL: https://issues.apache.org/jira/browse/NIFI-12221 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a heartbeat is received by the Cluster Coordinator, it responds based on > the node's current connection state. In the case of a disconnected node, it > either notifies the node that it is disconnected so that it will stop > hearting, or it requests the node to reconnect to the cluster. > Due to changes that were made in 1.16, as well as a few additional changes > that have been made since, we can be much more lenient about when we ask the > node to reconnect vs. disconnect. For example, if a node was disconnected due > to not handling an update request, we previously needed to request that the > node disconnect again. However, now we can ask the node to reconnect, as it > may well be able to reconcile any differences and rejoin. > We even currently request that a node disconnect if receiving a heartbeat > from a node whose last state was "Disconnected because Node was Shutdown". We > should definitely be more lenient in this case, as it's occasionally causing > System Test failures (e.g., > [https://github.com/apache/nifi/actions/runs/6498488206).|https://github.com/apache/nifi/actions/runs/6498488206)] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12221) Make heartbeat responses more lenient in some cases
[ https://issues.apache.org/jira/browse/NIFI-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775062#comment-17775062 ] ASF subversion and git services commented on NIFI-12221: Commit f4ae292a457638d3226fb0491e5186fa52ae8518 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f4ae292a45 ] NIFI-12221: This closes #7876. Be more lenient about which Disconnection Codes we allow a node to be reconnected to a cluster vs. when we notify the node to disconnect again. Also updated the timeout for OffloadIT because it occasionally times ou out while running properly. Signed-off-by: Joseph Witt > Make heartbeat responses more lenient in some cases > --- > > Key: NIFI-12221 > URL: https://issues.apache.org/jira/browse/NIFI-12221 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > When a heartbeat is received by the Cluster Coordinator, it responds based on > the node's current connection state. In the case of a disconnected node, it > either notifies the node that it is disconnected so that it will stop > hearting, or it requests the node to reconnect to the cluster. > Due to changes that were made in 1.16, as well as a few additional changes > that have been made since, we can be much more lenient about when we ask the > node to reconnect vs. disconnect. For example, if a node was disconnected due > to not handling an update request, we previously needed to request that the > node disconnect again. However, now we can ask the node to reconnect, as it > may well be able to reconcile any differences and rejoin. > We even currently request that a node disconnect if receiving a heartbeat > from a node whose last state was "Disconnected because Node was Shutdown". We > should definitely be more lenient in this case, as it's occasionally causing > System Test failures (e.g., > [https://github.com/apache/nifi/actions/runs/6498488206).|https://github.com/apache/nifi/actions/runs/6498488206)] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12221: Be more lenient about which Disconnection Codes we allow … [nifi]
asfgit closed pull request #7876: NIFI-12221: Be more lenient about which Disconnection Codes we allow … URL: https://github.com/apache/nifi/pull/7876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12228) Concurrency bug can occasionally lead to allowing a group with Single FlowFile per Node input pulling in multiple FlowFiles
[
https://issues.apache.org/jira/browse/NIFI-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-12228.
-
Fix Version/s: 2.0.0
1.24.0
(was: 1.latest)
(was: 2.latest)
Resolution: Fixed
> Concurrency bug can occasionally lead to allowing a group with Single
> FlowFile per Node input pulling in multiple FlowFiles
> ---
>
> Key: NIFI-12228
> URL: https://issues.apache.org/jira/browse/NIFI-12228
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0, 1.24.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Every now and then we see a failure in the system tests:
> {code:java}
> BatchFlowBetweenGroupsIT.testSingleConcurrencyAndBatchOutputToBatchInputOutput
> » Timeout testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out
> after 5 minutes {code}
> Looking at the logs shows that this is happening because data enters Group A,
> and then before the Output Port has a chance to push the data out of Group A,
> a second FlowFile enters. As a result, the test fails because the queue
> between Groups A and B, or the connection inside of Group B, never reach the
> expected size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasiona… [nifi]
joewitt closed pull request #7882: NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasiona… URL: https://github.com/apache/nifi/pull/7882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12228) Concurrency bug can occasionally lead to allowing a group with Single FlowFile per Node input pulling in multiple FlowFiles
[
https://issues.apache.org/jira/browse/NIFI-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775056#comment-17775056
]
ASF subversion and git services commented on NIFI-12228:
Commit 4d00f8e1563ffec40d8d601ac2b678ad0414bb5b in nifi's branch
refs/heads/support/nifi-1.x from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4d00f8e156 ]
NIFI-12228: This closes #7882. Fixed issue with FlowFile Concucrrency that can
occasionally bring in more data than it should.
Signed-off-by: Joseph Witt
> Concurrency bug can occasionally lead to allowing a group with Single
> FlowFile per Node input pulling in multiple FlowFiles
> ---
>
> Key: NIFI-12228
> URL: https://issues.apache.org/jira/browse/NIFI-12228
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Every now and then we see a failure in the system tests:
> {code:java}
> BatchFlowBetweenGroupsIT.testSingleConcurrencyAndBatchOutputToBatchInputOutput
> » Timeout testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out
> after 5 minutes {code}
> Looking at the logs shows that this is happening because data enters Group A,
> and then before the Output Port has a chance to push the data out of Group A,
> a second FlowFile enters. As a result, the test fails because the queue
> between Groups A and B, or the connection inside of Group B, never reach the
> expected size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasionally bring in more data than it should. [nifi]
asfgit closed pull request #7881: NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasionally bring in more data than it should. URL: https://github.com/apache/nifi/pull/7881 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12228) Concurrency bug can occasionally lead to allowing a group with Single FlowFile per Node input pulling in multiple FlowFiles
[
https://issues.apache.org/jira/browse/NIFI-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775055#comment-17775055
]
ASF subversion and git services commented on NIFI-12228:
Commit 0eabbcdf19dfa9de7085b4b173ac3d3260f9d3a4 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0eabbcdf19 ]
NIFI-12228: This closes #7881. Fixed issue with FlowFile Concucrrency that can
occasionally bring in more data than it should.
Code cleanup, fixing logback to avoid INFO-level stack trace from xodus
Signed-off-by: Joseph Witt
> Concurrency bug can occasionally lead to allowing a group with Single
> FlowFile per Node input pulling in multiple FlowFiles
> ---
>
> Key: NIFI-12228
> URL: https://issues.apache.org/jira/browse/NIFI-12228
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Every now and then we see a failure in the system tests:
> {code:java}
> BatchFlowBetweenGroupsIT.testSingleConcurrencyAndBatchOutputToBatchInputOutput
> » Timeout testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out
> after 5 minutes {code}
> Looking at the logs shows that this is happening because data enters Group A,
> and then before the Output Port has a chance to push the data out of Group A,
> a second FlowFile enters. As a result, the test fails because the queue
> between Groups A and B, or the connection inside of Group B, never reach the
> expected size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12175 update Docker Image builds to Java 21, rationalise base image name and tag across all components [nifi]
ChrisSamo632 commented on PR #7864: URL: https://github.com/apache/nifi/pull/7864#issuecomment-1762056399 This PR blocks NIFI-12178 (see #7858) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12178 add integration-tests and docker-tests github actions [nifi]
ChrisSamo632 commented on code in PR #7858: URL: https://github.com/apache/nifi/pull/7858#discussion_r1358711103 ## .github/workflows/docker-tests.yml: ## @@ -0,0 +1,189 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: docker-tests + +on: + # Run every day at 02:00 + schedule: +- cron: "0 2 * * *" + push: +paths: + - '.github/workflows/docker-tests.yml' + - 'nifi-assembly/**' + - 'nifi-docker/**' + - 'nifi-registry/nifi-registry-assembly/**' + - 'nifi-registry/nifi-registry-docker-maven/**' + - 'nifi-toolkit/nifi-toolkit-assembly/**' + - 'nifi-toolkit/nifi-toolkit-cli/**' + - 'nifi-toolkit/nifi-toolkit-encrypt-config/**' + - 'nifi-toolkit/nifi-toolkit-tls/**' + - 'minifi/minifi-assembly/**' + - 'minifi/minifi-docker/**' + - 'minifi/minifi-c2/minifi-c2-assembly/**' + - 'minifi/minifi-c2/minifi-c2-docker/**' + pull_request: +paths: + - '.github/workflows/docker-tests.yml' + - 'nifi-assembly/**' + - 'nifi-docker/**' + - 'nifi-registry/nifi-registry-assembly/**' + - 'nifi-registry/nifi-registry-docker-maven/**' + - 'nifi-toolkit/nifi-toolkit-assembly/**' + - 'nifi-toolkit/nifi-toolkit-cli/**' + - 'nifi-toolkit/nifi-toolkit-encrypt-config/**' + - 'nifi-toolkit/nifi-toolkit-tls/**' Review Comment: NIFI-12200 (see #7879) will lead to this needing to be changed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12178 add integration-tests and docker-tests github actions [nifi]
ChrisSamo632 commented on code in PR #7858:
URL: https://github.com/apache/nifi/pull/7858#discussion_r1358710047
##
.github/workflows/docker-tests.yml:
##
@@ -0,0 +1,189 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+name: docker-tests
+
+on:
+ # Run every day at 02:00
+ schedule:
+- cron: "0 2 * * *"
+ push:
+paths:
+ - '.github/workflows/docker-tests.yml'
+ - 'nifi-assembly/**'
+ - 'nifi-docker/**'
+ - 'nifi-registry/nifi-registry-assembly/**'
+ - 'nifi-registry/nifi-registry-docker-maven/**'
+ - 'nifi-toolkit/nifi-toolkit-assembly/**'
+ - 'nifi-toolkit/nifi-toolkit-cli/**'
+ - 'nifi-toolkit/nifi-toolkit-encrypt-config/**'
+ - 'nifi-toolkit/nifi-toolkit-tls/**'
+ - 'minifi/minifi-assembly/**'
+ - 'minifi/minifi-docker/**'
+ - 'minifi/minifi-c2/minifi-c2-assembly/**'
+ - 'minifi/minifi-c2/minifi-c2-docker/**'
+ pull_request:
+paths:
+ - '.github/workflows/docker-tests.yml'
+ - 'nifi-assembly/**'
+ - 'nifi-docker/**'
+ - 'nifi-registry/nifi-registry-assembly/**'
+ - 'nifi-registry/nifi-registry-docker-maven/**'
+ - 'nifi-toolkit/nifi-toolkit-assembly/**'
+ - 'nifi-toolkit/nifi-toolkit-cli/**'
+ - 'nifi-toolkit/nifi-toolkit-encrypt-config/**'
+ - 'nifi-toolkit/nifi-toolkit-tls/**'
+ - 'minifi/minifi-assembly/**'
+ - 'minifi/minifi-docker/**'
+ - 'minifi/minifi-c2/minifi-c2-assembly/**'
+ - 'minifi/minifi-c2/minifi-c2-docker/**'
+
+env:
+ DEFAULT_MAVEN_OPTS: >-
+-Xmx4g
+-XX:ReservedCodeCacheSize=1g
+-XX:+UseG1GC
+-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN
+-Daether.connector.http.retryHandler.count=5
+-Daether.connector.http.connectionMaxTtl=30
+ MAVEN_COMMAND: >-
+./mvnw
+-V
+-nsu
+-ntp
+-ff
+
+# build assemblies (including binaries) for use in Docker Image builds
+ MAVEN_BUILD_ARGUMENTS: >-
+-am
+-D skipTests
+-T 2.0C
+install
+ MAVEN_BUILD_PROJECTS: >-
+-pl nifi-assembly
+-pl nifi-registry/nifi-registry-assembly
+-pl nifi-toolkit/nifi-toolkit-assembly
+-pl minifi/minifi-assembly
+-pl minifi/minifi-c2/minifi-c2-assembly
+
+ MAVEN_CLEAN_ARGUMENTS: >-
+-T 2.0C
+clean
+ MAVEN_CLEAN_PROJECTS: >-
+-pl -nifi-assembly
+-pl -nifi-registry/nifi-registry-assembly
+-pl -nifi-toolkit/nifi-toolkit-assembly
+-pl -nifi-toolkit/nifi-toolkit-cli
+-pl -nifi-toolkit/nifi-toolkit-encrypt-config
+-pl -nifi-toolkit/nifi-toolkit-tls
+-pl -minifi/minifi-assembly
+-pl -minifi/minifi-c2/minifi-c2-assembly
+
+# build and test Docker Images
+ MAVEN_DOCKER_ARGUMENTS: >-
+package
+-P docker
+ MAVEN_DOCKER_PROJECTS: >-
+-pl minifi/minifi-docker
+-pl minifi/minifi-c2/minifi-c2-docker
+-pl nifi-docker/dockermaven
+-pl nifi-registry/nifi-registry-docker-maven/dockermaven
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+permissions:
+ contents: read
+
+jobs:
+ build_and_test:
+strategy:
+ fail-fast: false
+ matrix:
+# TODO: macos-latest doesn't come with Docker and Colima seems to have
problems with the nifi-docker integration_test scripts - see
NIFI-12177/NIFI-12191
+os: [ ubuntu-latest ]
+version: [ 21 ]
+timeout-minutes: 120
+runs-on: ${{ matrix.os }}
+name: ${{ matrix.os }} Java ${{ matrix.version }}
+env:
+ # default to Amazon Corretto JDK
+ JAVA_DISTRIBUTION: corretto
+steps:
+ # Zulu JDK appears to be more reliable for MacOS
+ - name: Use Java distribution Zulu
+if: ${{ runner.os == 'macOS' }}
+run: echo "JAVA_DISTRIBUTION=zulu" >> "$GITHUB_ENV"
+ - name: Checkout Code
+uses: actions/checkout@v4
+ - name: Set up Java ${{ env.JAVA_DISTRIBUTION }} ${{ matrix.version }}
+uses: actions/setup-java@v3
+with:
+ distribution: ${{ env.JAVA_DISTRIBUTION }}
+ java-version: ${{ matrix.version }}
+ cache: 'maven'
+
+ - name: Set up Docker
+if: ${{ runner.os == 'macOS' }}
+run: |-
+ /bin/bash -c "$(curl -fsSL
Re: [PR] NIFI-11197 Initial check in for Yaml record reader [nifi]
dan-s1 commented on PR #7665: URL: https://github.com/apache/nifi/pull/7665#issuecomment-1762022382 @exceptionfactory I am trying to rework my code to handle the changes made in #7823 for NIFI-12153. What I am discovering is that YAML supports comments and there is no way to turn off parsing of comments. In addition I am trying to mimic `TestJsonTreeRowRecordReader#testReadJSONStringTooLong` and I am trying to use `StreamReadConstraints.builder().maxStringLength(1).build()` yet no exception is thrown even though I have strings longer than one character long. It would seem that aspect only works for Json parsing and not Yaml parsing. I did find the following article [How to parse large YAML file in Java or Kotlin](https://stackoverflow.com/questions/74805240/how-to-parse-large-yaml-file-in-java-or-kotlin) which would seem to handle larger Yaml files but I am not sure if that is the same setting as `StreamReadConstraints.builder().maxStringLength(1).build()`. Please advise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12030 Property pop-up window is horizontally resizable [nifi]
annanys23 opened a new pull request, #7883: URL: https://github.com/apache/nifi/pull/7883 # Summary [NIFI-12030](https://issues.apache.org/jira/browse/NIFI-12030) When a property of a controller service or processor is a list of values, the combo box is now horizontally resizable when open. # Tracking ### Issue Tracking - [x ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [x] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12222) NPE in StandardVersionedComponentSynchronizer if PG references missing param context
[
https://issues.apache.org/jira/browse/NIFI-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-1:
--
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NPE in StandardVersionedComponentSynchronizer if PG references missing param
> context
>
>
> Key: NIFI-1
> URL: https://issues.apache.org/jira/browse/NIFI-1
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.2
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> java.lang.NullPointerException: null
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateParameterContext(StandardVersionedComponentSynchronizer.java:1963)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:301)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1203)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:545)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:431)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:262)
> at
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:550)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:257)
> at
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3986)
> at
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3966)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12222) NPE in StandardVersionedComponentSynchronizer if PG references missing param context
[
https://issues.apache.org/jira/browse/NIFI-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775008#comment-17775008
]
ASF subversion and git services commented on NIFI-1:
Commit 96eb1d825a0f23f7b13c4a1f31853b0cef10b7bc in nifi's branch
refs/heads/main from Bryan Bende
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=96eb1d825a ]
NIFI-1 Protect against missing parameter context when syncing a PG in
component synchronizer (#7877)
> NPE in StandardVersionedComponentSynchronizer if PG references missing param
> context
>
>
> Key: NIFI-1
> URL: https://issues.apache.org/jira/browse/NIFI-1
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.2
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> java.lang.NullPointerException: null
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateParameterContext(StandardVersionedComponentSynchronizer.java:1963)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:301)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1203)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:545)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:431)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:262)
> at
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:550)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:257)
> at
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3986)
> at
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3966)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12222 Protect against missing parameter context when syncing a P… [nifi]
markap14 commented on PR #7877: URL: https://github.com/apache/nifi/pull/7877#issuecomment-1761945351 Thanks for fixing @bbende. +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12222 Protect against missing parameter context when syncing a P… [nifi]
markap14 merged PR #7877: URL: https://github.com/apache/nifi/pull/7877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12229) Edits to Registry-Client in NiFi require NiFi service restart to take affect.
Matthew Clarke created NIFI-12229: - Summary: Edits to Registry-Client in NiFi require NiFi service restart to take affect. Key: NIFI-12229 URL: https://issues.apache.org/jira/browse/NIFI-12229 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.23.1 Reporter: Matthew Clarke Within NiFi --> global menu --> Controller Settings --> Registry Clients there is no option to "disable" and "enable" the added registry clients. The ability to edit a Registry Client is allowed and there is even a button "Update" a user can click on after making edits. However, clicking "update" does not apply the changes. A restart of the NiFi service is required for updates to take affect. How to reproduce - Create a StandardRestrictedSSLContextService configured with only a truststore - Create a second StandardRestrictedSSLContextService configured with both keystore and truststore. - Create a Registry Client (NiFiRegistryFlowRegistryClient) and configure https:/: and configure to use SSL Context service created with both keystore and truststore - From canvas start version control on a process group. - Now edit the registry client so that it uses SSL context service with only truststore and click "Update". - Return to canvas and edit version controlled PG and commit new version. You'll notice this is still possible because registry client continues to use the SSL Context service that has both keystore and truststore. - If you then restart the NiFi service, you'll notice the version controlled PG now exhibit a "?" because it is unable to retrieve version flow information from Registry because it is now loaded with different ssl context service. NiFi-Registry logs will show "anonymous" access attempts as expected. While above allows for easy method to reproduce issue. The more likely issue will occur when a user obtains new certificates and tries to update an existing SSL Context service or create a new and then tries to use it in the Registry Client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12124: Added a new RenameRecordField processor. In testing, also… [nifi]
ChrisSamo632 commented on PR #7791: URL: https://github.com/apache/nifi/pull/7791#issuecomment-1761759656 I'll try to take a look through if possible (but don't wait for me if someone else reviews first) I just noticed this and wondered whether it also addresses [NIFI-7194](https://issues.apache.org/jira/browse/NIFI-7194) as well - not suggesting it needs to be updated to do so if the changes don't already provide the suggested improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12200 - Remove deprecated module nifi-toolkit-tls [nifi]
ChrisSamo632 commented on code in PR #7879: URL: https://github.com/apache/nifi/pull/7879#discussion_r1358442867 ## nifi-toolkit/pom.xml: ## @@ -23,7 +23,6 @@ nifi-toolkit pom -nifi-toolkit-tls Review Comment: The `tls-toolkit` is also referenced by the `nifi-toolkit/nifi-toolkit-assembly`'s `docker` build and tests, it will need removing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774947#comment-17774947
]
Mark Payne commented on NIFI-11627:
---
[~nwchuckster] yes, the approach that you mentioned there, which uses
{{evaluateELString}} was an oversight that has since been patched. Anywhere
that you're using that, you'll want to fix because in more recent versions that
will fail.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasiona… [nifi]
markap14 opened a new pull request, #7882: URL: https://github.com/apache/nifi/pull/7882 …lly bring in more data than it should. # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12228: Fixed issue with FlowFile Concucrrency that can occasionally bring in more data than it should. [nifi]
markap14 opened a new pull request, #7881: URL: https://github.com/apache/nifi/pull/7881 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12228) Concurrency bug can occasionally lead to allowing a group with Single FlowFile per Node input pulling in multiple FlowFiles
[
https://issues.apache.org/jira/browse/NIFI-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774938#comment-17774938
]
Mark Payne commented on NIFI-12228:
---
Because this is a concurrent bug, it is difficult to replicate/verify. However,
I was able to replicate the issue locally by changing the code slightly. In
{{{}LocalPort{}}}, I modified the {{triggerOutputPort}} to sleep before closing
flow out of the group:
{code:java}
try {
transferUnboundedConcurrency(context, session);
} finally {
try {
Thread.sleep(1500L);
} catch (final InterruptedException ignored) {
}
dataValve.closeFlowOutOfGroup(getProcessGroup());
}{code}
With this in place, after rebuilding, the system tests failed in the same
manner. This is because the output port has a chance to run and transfer data
out of the group. But before it closes the valve, the Input Port now has a
chance to run and open the gate for allowing data to flow into the group. It
then pulls a FlowFile in before the output valve is closed.
As a result, the call to {{dataValve.closeFlowOutOfGroup}} returns {{false}}
because the group is not empty. As a result, the FlowFiles in the connection
between Groups A and B cannot be access (because Group B's output valve is left
open). This, in turn, means that the valve won't be closed until the next group
fo data is pushed out. Now there are 10 FlowFiles in the connection between
Groups A and B, not 5. So the test times out waiting for there to be 5
FlowFiles. Meanwhile, Group A keeps bringing new data in until eventually there
are 25 FlowFiles in that connection. But since the test is expecting exactly 5,
it times out waiting for that to occur.
> Concurrency bug can occasionally lead to allowing a group with Single
> FlowFile per Node input pulling in multiple FlowFiles
> ---
>
> Key: NIFI-12228
> URL: https://issues.apache.org/jira/browse/NIFI-12228
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.latest, 2.latest
>
>
> Every now and then we see a failure in the system tests:
> {code:java}
> BatchFlowBetweenGroupsIT.testSingleConcurrencyAndBatchOutputToBatchInputOutput
> » Timeout testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out
> after 5 minutes {code}
> Looking at the logs shows that this is happening because data enters Group A,
> and then before the Output Port has a chance to push the data out of Group A,
> a second FlowFile enters. As a result, the test fails because the queue
> between Groups A and B, or the connection inside of Group B, never reach the
> expected size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12228) Concurrency bug can occasionally lead to allowing a group with Single FlowFile per Node input pulling in multiple FlowFiles
Mark Payne created NIFI-12228:
-
Summary: Concurrency bug can occasionally lead to allowing a group
with Single FlowFile per Node input pulling in multiple FlowFiles
Key: NIFI-12228
URL: https://issues.apache.org/jira/browse/NIFI-12228
Project: Apache NiFi
Issue Type: Bug
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 1.latest, 2.latest
Every now and then we see a failure in the system tests:
{code:java}
BatchFlowBetweenGroupsIT.testSingleConcurrencyAndBatchOutputToBatchInputOutput
» Timeout testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out
after 5 minutes {code}
Looking at the logs shows that this is happening because data enters Group A,
and then before the Output Port has a chance to push the data out of Group A, a
second FlowFile enters. As a result, the test fails because the queue between
Groups A and B, or the connection inside of Group B, never reach the expected
size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12201) Deprecation markings for nifi-toolkit-tls module in support/nifi-1.x
[ https://issues.apache.org/jira/browse/NIFI-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-12201: - Status: Patch Available (was: In Progress) > Deprecation markings for nifi-toolkit-tls module in support/nifi-1.x > > > Key: NIFI-12201 > URL: https://issues.apache.org/jira/browse/NIFI-12201 > Project: Apache NiFi > Issue Type: Task >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Module nifi-toolkit-tls functionality should be marked deprecated in NiFi 1.x > branch. > Example marking: > https://github.com/apache/nifi/blob/support/nifi-1.x/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1226 > Possible home for marking: > https://github.com/apache/nifi/blob/support/nifi-1.x/nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/TlsToolkitMain.java#L88 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12201 - Deprecation marking for nifi-toolkit-tls in NiFi 1.x [nifi]
greyp9 opened a new pull request, #7880: URL: https://github.com/apache/nifi/pull/7880 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12200) Remove nifi-toolkit-tls module
[ https://issues.apache.org/jira/browse/NIFI-12200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-12200: - Status: Patch Available (was: In Progress) > Remove nifi-toolkit-tls module > -- > > Key: NIFI-12200 > URL: https://issues.apache.org/jira/browse/NIFI-12200 > Project: Apache NiFi > Issue Type: Task >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > mailing list discussion: > https://lists.apache.org/thread/vn1nzobtz4fh7fs461sgg8jj9zygrk0f > Deprecate TLS Toolkit for targeted NiFi 2.0 release, in order to address > maintenance concerns. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12200 - Remove deprecated module nifi-toolkit-tls [nifi]
greyp9 opened a new pull request, #7879: URL: https://github.com/apache/nifi/pull/7879 # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (NIFI-12227) Update Zookeeper to 3.9.1
[ https://issues.apache.org/jira/browse/NIFI-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R closed NIFI-12227. - > Update Zookeeper to 3.9.1 > - > > Key: NIFI-12227 > URL: https://issues.apache.org/jira/browse/NIFI-12227 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update Zookeeper to 3.9.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12227) Update Zookeeper to 3.9.1
[ https://issues.apache.org/jira/browse/NIFI-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R resolved NIFI-12227. --- Resolution: Fixed Already done > Update Zookeeper to 3.9.1 > - > > Key: NIFI-12227 > URL: https://issues.apache.org/jira/browse/NIFI-12227 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update Zookeeper to 3.9.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12227) Update Zookeeper to 3.9.1
[ https://issues.apache.org/jira/browse/NIFI-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-12227: - Assignee: Mike R > Update Zookeeper to 3.9.1 > - > > Key: NIFI-12227 > URL: https://issues.apache.org/jira/browse/NIFI-12227 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update Zookeeper to 3.9.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12227) Update Zookeeper to 3.9.1
Mike R created NIFI-12227: - Summary: Update Zookeeper to 3.9.1 Key: NIFI-12227 URL: https://issues.apache.org/jira/browse/NIFI-12227 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.23.2 Reporter: Mike R Update Zookeeper to 3.9.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12225 Improve NiFi splash loading animation [nifi]
nandorsoma opened a new pull request, #7878: URL: https://github.com/apache/nifi/pull/7878 # Summary [NIFI-12225](https://issues.apache.org/jira/browse/NIFI-12225) With NiFi 2.0 just around the corner, I thought it's a great opportunity to differentiate it from from the 1.x line by introducing a fresh splash screen. The PR includes both a GIF and a JSON version. The JSON format would be the preferred choice, but it requires an additional frontend dependency to render. Personally, I don't believe that having just a single animation from that source justifies adding the extra dependency. I'm including it in the PR primarily as an option for potential future use. To see it in action, I recommend setting a breakpoint at: https://github.com/apache/nifi/blob/ae61ebb5eda7076c6a18ba0419b614b8516faf14/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java#L1229 Screenshot taken from a running NiFi: https://github.com/apache/nifi/assets/11146937/9d302bfe-3123-4875-93ef-8aa2730a3b0d # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12226) Maven plugin: Add flag to skip doc generation
[ https://issues.apache.org/jira/browse/NIFI-12226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolò Boschi updated NIFI-12226: - Description: In [LangStream/langstream|https://github.com/LangStream/langstream] we use this plugin for producing NAR files. The NAR are then ingested by the LangStream runtime and the documentation is not needed. > Maven plugin: Add flag to skip doc generation > - > > Key: NIFI-12226 > URL: https://issues.apache.org/jira/browse/NIFI-12226 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nicolò Boschi >Priority: Minor > > In [LangStream/langstream|https://github.com/LangStream/langstream] we use > this plugin for producing NAR files. The NAR are then ingested by the > LangStream runtime and the documentation is not needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12226) Maven plugin: Add flag to skip doc generation
[ https://issues.apache.org/jira/browse/NIFI-12226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolò Boschi updated NIFI-12226: - Priority: Minor (was: Major) > Maven plugin: Add flag to skip doc generation > - > > Key: NIFI-12226 > URL: https://issues.apache.org/jira/browse/NIFI-12226 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nicolò Boschi >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12226: Add flag to skip doc generation [nifi-maven]
nicoloboschi commented on PR #35: URL: https://github.com/apache/nifi-maven/pull/35#issuecomment-1761300835 > Do you mind filing a NIFI JIRA for this change? done https://issues.apache.org/jira/browse/NIFI-12226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12226) Maven plugin: Add flag to skip doc generation
Nicolò Boschi created NIFI-12226: Summary: Maven plugin: Add flag to skip doc generation Key: NIFI-12226 URL: https://issues.apache.org/jira/browse/NIFI-12226 Project: Apache NiFi Issue Type: Improvement Reporter: Nicolò Boschi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12225) Improve NiFi splash loading animation
[ https://issues.apache.org/jira/browse/NIFI-12225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Soma Abonyi updated NIFI-12225: -- Description: With NiFi 2.0 just around the corner, I thought it's a great opportunity to differentiate it from from the 1.x line by introducing a fresh splash screen. The PR includes both a GIF and a JSON version. The JSON format would be the preferred choice, but it requires an additional frontend dependency to render. Personally, I don't believe that having just a single animation from that source justifies adding the extra dependency. I'm including it in the PR primarily as an option for potential future use. was:With NiFi 2.0 just around the corner, I thought it's a great opportunity to differentiate it from from the 1.x line by introducing a fresh splash screen. > Improve NiFi splash loading animation > - > > Key: NIFI-12225 > URL: https://issues.apache.org/jira/browse/NIFI-12225 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 2.latest > > > With NiFi 2.0 just around the corner, I thought it's a great opportunity to > differentiate it from from the 1.x line by introducing a fresh splash screen. > The PR includes both a GIF and a JSON version. The JSON format would be the > preferred choice, but it requires an additional frontend dependency to > render. Personally, I don't believe that having just a single animation from > that source justifies adding the extra dependency. I'm including it in the PR > primarily as an option for potential future use. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12225) Improve NiFi splash loading animation
Nandor Soma Abonyi created NIFI-12225: - Summary: Improve NiFi splash loading animation Key: NIFI-12225 URL: https://issues.apache.org/jira/browse/NIFI-12225 Project: Apache NiFi Issue Type: Improvement Reporter: Nandor Soma Abonyi Assignee: Nandor Soma Abonyi With NiFi 2.0 just around the corner, I thought it's a great opportunity to differentiate it from from the 1.x line by introducing a fresh splash screen. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12225) Improve NiFi splash loading animation
[ https://issues.apache.org/jira/browse/NIFI-12225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Soma Abonyi updated NIFI-12225: -- Fix Version/s: 2.latest > Improve NiFi splash loading animation > - > > Key: NIFI-12225 > URL: https://issues.apache.org/jira/browse/NIFI-12225 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 2.latest > > > With NiFi 2.0 just around the corner, I thought it's a great opportunity to > differentiate it from from the 1.x line by introducing a fresh splash screen. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12194) Nifi fails when ConsumeKafka_2_6 processor is started with PLAINTEXT securityProtocol
[ https://issues.apache.org/jira/browse/NIFI-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Schmitzer updated NIFI-12194: --- Affects Version/s: 1.23.0 > Nifi fails when ConsumeKafka_2_6 processor is started with PLAINTEXT > securityProtocol > - > > Key: NIFI-12194 > URL: https://issues.apache.org/jira/browse/NIFI-12194 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.21.0, 1.23.0 >Reporter: Peter Schmitzer >Priority: Major > Attachments: image-2023-09-27-15-56-02-438.png > > > When starting ConsumeKafka_2_6 processor with sasl mechanism GSSAPI and the > securityProtocol PLAINTEXT (although SSL would be correct) the UI crashed and > nifi was no longer accessible. Not only the frontend was not accessible > anymore, also the other processors in our flow stopped performing well > according to our dashboards. > We were able to reproduce this by using the config as described above. > Our nifi in preprod (where this was detected) runs in a kubernetes cluster. > * version 1.21.0 > * 3 nodes > * jvmMemory: 1536m > * 3G memory (limit) > * 400m cpu (request) > * zookeeper > The logs do not offer any unusual entries when the issue is triggered. > Inspecting the pod metrics we found a spike in memory. > The issue is a bit scary for us because a rather innocent config parameter in > one single processor is able to let our whole cluster break down. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12224) PutMongo with operator only updates the first match
Malte created NIFI-12224: Summary: PutMongo with operator only updates the first match Key: NIFI-12224 URL: https://issues.apache.org/jira/browse/NIFI-12224 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.23.2 Reporter: Malte Problem to solve: In my case, I wanted to add a field to a couple of documents that fullfill a certain condition. The [PutMongo documentation|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.23.0/org.apache.nifi.processors.mongodb.PutMongo/additionalDetails.html] states that if you select the "With Operators Enabled" mode, then the "Update Query" property can be used "to find one or more documents to update". But the processor uses the ["updateOne" method of the MongoCollection class|https://github.com/apache/nifi/blob/22ad7d542d627e767f962b13236da90a0d6410f5/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java#L230C21-L230C41], which will only update the first match. Proposed solutions: 1. Update the documentation to correctly state that only one document (the first match) can be updated using an operator. 2. Use the updateMany method of the MongoCollection class. 3. Add a property like ["update-mode" from PutMongoRecord|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.23.0/org.apache.nifi.processors.mongodb.PutMongoRecord/] to switch between updateOne and updateMany. Workaround: Right now I use the RunMongoAggregation and use a $match, $addFields and $merge operation to achieve what I want. But I think the PutMongo processor would be the intuitive way to go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] Add flag to skip doc generation [nifi-maven]
pvillard31 commented on PR #35: URL: https://github.com/apache/nifi-maven/pull/35#issuecomment-1761123734 Do you mind filing a NIFI JIRA for this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add flag to skip doc generation [nifi-maven]
nandorsoma commented on PR #35: URL: https://github.com/apache/nifi-maven/pull/35#issuecomment-1761020269 > @nandorsoma @kevdoran could you review? > > > > it would be great to cut a release right after Sure @nicoloboschi, thanks for the heads-up! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add flag to skip doc generation [nifi-maven]
nicoloboschi commented on PR #35: URL: https://github.com/apache/nifi-maven/pull/35#issuecomment-1761008569 @nandorsoma @kevdoran could you review? it would be great to cut a release right after -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Add flag to skip doc generation [nifi-maven]
nicoloboschi opened a new pull request, #35: URL: https://github.com/apache/nifi-maven/pull/35 In https://github.com/LangStream/langstream we use this plugin for producing NAR files. The NAR are then ingested by the LangStream runtime and the documentation is not needed. Changes: * Added new boolean configuration flag "skipDocGeneration" (default false) to completely skip the doc generation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
