[jira] [Updated] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12858: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1, 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 20m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12858 Correct Order of Previous Property Values [nifi]
asfgit closed pull request #8673: NIFI-12858 Correct Order of Previous Property Values URL: https://github.com/apache/nifi/pull/8673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839177#comment-17839177 ] ASF subversion and git services commented on NIFI-12858: Commit 64e99f0166ac3574604e10e3ed4191bfa139091c in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=64e99f0166 ] NIFI-12858 Corrected Order of Previous Property Values - Reversed default order of previous values to return oldest entries before newest entries Signed-off-by: Pierre Villard This closes #8673. > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1, 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 20m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839176#comment-17839176 ] ASF subversion and git services commented on NIFI-12858: Commit 259a734a4b882343efd7c660fda829dcfffedc0c in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=259a734a4b ] NIFI-12858 Corrected Order of Previous Property Values - Reversed default order of previous values to return oldest entries before newest entries Signed-off-by: Pierre Villard This closes #8673. > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1, 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 10m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12998 Restructured project/build to drive better enforcement/maintainability of framework vs extensions. Makes it easier for people to build extensions outside our codebase as well with
joewitt commented on PR #8677: URL: https://github.com/apache/nifi/pull/8677#issuecomment-2067513494 So far unable to understand why github ci failed the nifi-jetty-nar test. Locally it works fine and the source that creates those nar's does not reference nifi-jetty-bundle or nifi-jetty-nar. What that is old and should be eliminated as a binary in our source tree our current full built works, all tests pass including the ci integration tests, system tests, and nifi runs. So not sure what the deal is there. Will focus on the security related version bumps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-12923. - Resolution: Fixed > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923 ] Joe Witt deleted comment on NIFI-12923: - was (Author: joewitt): [~balazsgerner][~mattyb149] I am reverting the commit for this test change as the two tests are unstable and need review/resolution. {noformat} [ERROR] Tests run: 20, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.478 s <<< FAILURE! -- in org.apache.nifi.processors.hadoop.PutHDFSTest [ERROR] org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeNewFileCreated -- Time elapsed: 0.005 s <<< ERROR! java.io.FileNotFoundException: src/test/resources/testdata-avro/input.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:213) at java.base/java.io.FileInputStream.(FileInputStream.java:152) at java.base/java.io.FileInputStream.(FileInputStream.java:106) at org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeNewFileCreated(PutHDFSTest.java:281) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) [ERROR] org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeWhenTargetFileAlreadyExists -- Time elapsed: 0.009 s <<< ERROR! java.io.FileNotFoundException: src/test/resources/testdata-avro/input.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:213) at java.base/java.io.FileInputStream.(FileInputStream.java:152) at java.base/java.io.FileInputStream.(FileInputStream.java:106) at org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeWhenTargetFileAlreadyExists(PutHDFSTest.java:308) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) {noformat} > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839172#comment-17839172 ] Joe Witt commented on NIFI-12923: - Disregard - error was in rebase on my end > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt reopened NIFI-12923: - > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839171#comment-17839171 ] Joe Witt commented on NIFI-12923: - [~balazsgerner][~mattyb149] I am reverting the commit for this test change as the two tests are unstable and need review/resolution. {noformat} [ERROR] Tests run: 20, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.478 s <<< FAILURE! -- in org.apache.nifi.processors.hadoop.PutHDFSTest [ERROR] org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeNewFileCreated -- Time elapsed: 0.005 s <<< ERROR! java.io.FileNotFoundException: src/test/resources/testdata-avro/input.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:213) at java.base/java.io.FileInputStream.(FileInputStream.java:152) at java.base/java.io.FileInputStream.(FileInputStream.java:106) at org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeNewFileCreated(PutHDFSTest.java:281) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) [ERROR] org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeWhenTargetFileAlreadyExists -- Time elapsed: 0.009 s <<< ERROR! java.io.FileNotFoundException: src/test/resources/testdata-avro/input.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:213) at java.base/java.io.FileInputStream.(FileInputStream.java:152) at java.base/java.io.FileInputStream.(FileInputStream.java:106) at org.apache.nifi.processors.hadoop.PutHDFSTest.testPutFileWithAppendAvroModeWhenTargetFileAlreadyExists(PutHDFSTest.java:308) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) {noformat} > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13065) Self looping connection missing bend points
[ https://issues.apache.org/jira/browse/NIFI-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839130#comment-17839130 ] ASF subversion and git services commented on NIFI-13065: Commit d9e48f8645a66cd9088b6566e706dfdda775e300 in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d9e48f8645 ] NIFI-13065: Adding initial bend points for self looping connections and connections that collide with existing connections (#8671) * NIFI-13065: - Adding initial bend points for self looping connections and connections that collide with existing connections. - Merging two actions into one for opening the new connection dialog. * NIFI-13065: - Only considering self looping connections when automatically moving bends when the source component moves. * NIFI-13065: - Making collision check for more lenient. - Setting initial label index to 0. This closes #8671 > Self looping connection missing bend points > --- > > Key: NIFI-13065 > URL: https://issues.apache.org/jira/browse/NIFI-13065 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > When adding a self looping connection the bend points are missing. Also, when > connecting a source and destination that already have a connection, we are no > longer inserting and shifting bend points to avoid overlapping. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13065) Self looping connection missing bend points
[ https://issues.apache.org/jira/browse/NIFI-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-13065: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Self looping connection missing bend points > --- > > Key: NIFI-13065 > URL: https://issues.apache.org/jira/browse/NIFI-13065 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When adding a self looping connection the bend points are missing. Also, when > connecting a source and destination that already have a connection, we are no > longer inserting and shifting bend points to avoid overlapping. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13065: Adding initial bend points for self looping connections and connections that collide with existing connections [nifi]
rfellows merged PR #8671: URL: https://github.com/apache/nifi/pull/8671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13054) Tooltip Improvements
[ https://issues.apache.org/jira/browse/NIFI-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13054: --- Status: Patch Available (was: Reopened) > Tooltip Improvements > > > Key: NIFI-13054 > URL: https://issues.apache.org/jira/browse/NIFI-13054 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 40m > Remaining Estimate: 0h > > Fix z-index issue with tooltips that hover over buttons. Currently the button > text is above the tooltip tip. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12998 Restructured project/build to drive better enforcement/maintainability of framework vs extensions. Makes it easier for people to build extensions outside our codebase as well with
joewitt commented on PR #8677: URL: https://github.com/apache/nifi/pull/8677#issuecomment-2067242412 'nifi-jetty-bundle' was also renamed to 'nifi-jetty-nar' to be consistent with all other nar naming -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12998 Restructured project/build to drive better enforcement/maintainability of framework vs extensions. Makes it easier for people to build extensions outside our codebase as well with
joewitt commented on PR #8677: URL: https://github.com/apache/nifi/pull/8677#issuecomment-2067219879 1. need to fix dependency regressions that now are vuln versions. 2. need to fix errors in https://github.com/apache/nifi/actions/runs/8758685134/job/24040051585?pr=8677. This stems from us using pre-built nars in our test images. We should likely stop this. Note to reviewer: The PR is large but in major part this is true to directory creation/renames. What used to be in nifi-nar-bundles is now in two directories. All contents were moved from nifi-nar-bundles to nifi-extension-bundles All contents in nifi-nar-bundles/nifi-framework-bundle were moved to the top level. This was a key part of separating the build of the core framework from the build of all extensions. There are other minor directory creations for new bom(s) but otherwise the structure is in tact. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13075) AzureLogAnalyticsReportingTask does not re-initialize properly
[ https://issues.apache.org/jira/browse/NIFI-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-13075: Description: apache nifi slack user reports: The reporting task runs initially but when stopped and started again, it no longer processes provenance events until nifi is restarted. I figured by comparing the implementation to the SiteToSiteProvenance reporting tasks that it's because the event consumer isn't properly re-initialized [(this if statement is the problem)|https://github.com/apache/nifi/blob/6ecc398d3f92425447e43242af4992757e25b3c5/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-reporting-task/src/main/java/org/apache/nifi/reporting/azure/loganalytics/AzureLogAnalyticsProvenanceReportingTask.java#L201]. The SiteToSite task always re-initializes the event consumer always in onScheduled. Additionally, not a bug but an enhancement for the AzureLogAnalyticsProvenanceReportingTask is to keep state of the last processed event id, like SiteToSite does. AzureLogAnalyticsProvenanceReportingTask.java if (consumer != null) A bit more insight: the reporting task calls consumer.setScheduled(false) in onUnscheduled, but in onTrigger, due to aforementioned if statement, it never calls consumer.setScheduled(true), so this might the root cause. was: apache nifi slack user reports: The reporting task runs initially but when stopped and started again, it no longer processes provenance events until nifi is restarted. I figured by comparing the implementation to the SiteToSiteProvenance reporting tasks that it's because the event consumer isn't properly re-initialized [(this if statement is the problem)|https://github.com/apache/nifi/blob/6ecc398d3f92425447e43242af4992757e25b3c5/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-reporting-task/src/main/java/org/apache/nifi/reporting/azure/loganalytics/AzureLogAnalyticsProvenanceReportingTask.java#L201]. The SiteToSite task always re-initializes the event consumer always in onScheduled. Additionally, not a bug but an enhancement for the AzureLogAnalyticsProvenanceReportingTask is to keep state of the last processed event id, like SiteToSite does. AzureLogAnalyticsProvenanceReportingTask.java if (consumer != null) > AzureLogAnalyticsReportingTask does not re-initialize properly > -- > > Key: NIFI-13075 > URL: https://issues.apache.org/jira/browse/NIFI-13075 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joe Witt >Priority: Major > > apache nifi slack user reports: > The reporting task runs initially but when stopped and started again, it no > longer processes provenance events until nifi is restarted. I figured by > comparing the implementation to the SiteToSiteProvenance reporting tasks that > it's because the event consumer isn't properly re-initialized [(this if > statement is the > problem)|https://github.com/apache/nifi/blob/6ecc398d3f92425447e43242af4992757e25b3c5/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-reporting-task/src/main/java/org/apache/nifi/reporting/azure/loganalytics/AzureLogAnalyticsProvenanceReportingTask.java#L201]. > The SiteToSite task always re-initializes the event consumer always in > onScheduled. > Additionally, not a bug but an enhancement for the > AzureLogAnalyticsProvenanceReportingTask is to keep state of the last > processed event id, like SiteToSite does. > AzureLogAnalyticsProvenanceReportingTask.java > if (consumer != null) > A bit more insight: the reporting task calls consumer.setScheduled(false) in > onUnscheduled, but in onTrigger, due to aforementioned if statement, it never > calls consumer.setScheduled(true), so this might the root cause. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12998 Restructured project/build to drive better enforcement/maintainability of framework vs extensions. Makes it easier for people to build extensions outside our codebase as well with
joewitt commented on PR #8677: URL: https://github.com/apache/nifi/pull/8677#issuecomment-2067197169 @exceptionfactory reports the following deps show up now and had been upgraded. they have known vulns Specifically the following: commons-compiler-3.1.9.jar commons-configuration-1.10.jar janino-3.1.9.jar neo4j-1.19.4.jar nimbus-jose-jwt-9.31.jar Addressing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13075) AzureLogAnalyticsReportingTask does not re-initialize properly
Joe Witt created NIFI-13075: --- Summary: AzureLogAnalyticsReportingTask does not re-initialize properly Key: NIFI-13075 URL: https://issues.apache.org/jira/browse/NIFI-13075 Project: Apache NiFi Issue Type: Bug Reporter: Joe Witt apache nifi slack user reports: The reporting task runs initially but when stopped and started again, it no longer processes provenance events until nifi is restarted. I figured by comparing the implementation to the SiteToSiteProvenance reporting tasks that it's because the event consumer isn't properly re-initialized [(this if statement is the problem)|https://github.com/apache/nifi/blob/6ecc398d3f92425447e43242af4992757e25b3c5/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-reporting-task/src/main/java/org/apache/nifi/reporting/azure/loganalytics/AzureLogAnalyticsProvenanceReportingTask.java#L201]. The SiteToSite task always re-initializes the event consumer always in onScheduled. Additionally, not a bug but an enhancement for the AzureLogAnalyticsProvenanceReportingTask is to keep state of the last processed event id, like SiteToSite does. AzureLogAnalyticsProvenanceReportingTask.java if (consumer != null) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12998 Restructured project/build to drive better enforcement/maintainability of framework vs extensions. Makes it easier for people to build extensions outside our codebase as well with bom
joewitt opened a new pull request, #8677: URL: https://github.com/apache/nifi/pull/8677 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8063) Add profile to Maven POM to enable NAR exclusion
[ https://issues.apache.org/jira/browse/NIFI-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8063: --- Status: Open (was: Patch Available) > Add profile to Maven POM to enable NAR exclusion > > > Key: NIFI-8063 > URL: https://issues.apache.org/jira/browse/NIFI-8063 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Sometimes a bare-bones NiFi is all that is needed to test systems, > integrations, etc. It would be nice to be able to build a version of NiFi > without all the NARs in it (currently 1.5+ GB in total size). If this is done > as a profile in the assembly POM, the resulting artifacts can also be > dockerized (using the dockermaven module in the codebase). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-7252) wrong table mapping when using the nifi CaptureChangeMySQL modules combinded with mysql triggers:
[ https://issues.apache.org/jira/browse/NIFI-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-7252. Resolution: Duplicate > wrong table mapping when using the nifi CaptureChangeMySQL modules combinded > with mysql triggers: > - > > Key: NIFI-7252 > URL: https://issues.apache.org/jira/browse/NIFI-7252 > Project: Apache NiFi > Issue Type: Bug > Environment: Linux >Reporter: Michel Elias >Priority: Major > > wrong table mapping when using the nifi CaptureChangeMySQL modules combinded > with mysql triggers: > when using the nifi CaptureChangeMySQL module > ([https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQL.java]) > combinded with mysql triggers the table mapping does not work correctly. > when using triggers the order of the events is slightly different than when > using regular inserts. > == > regular insert: > BEGIN; > PREPARE stmt1 FROM 'insert into changes set Header_RefID=?'; > PREPARE stmt2 FROM 'insert into tab_tracedata set ZE140_RT_Aufnahme=?'; > SET @a = 3; > SET @b = 4; > EXECUTE stmt1 USING @a; > EXECUTE stmt2 USING @b; > COMMIT; > events in binlog: > |mysql-bin.29|349004|Gtid|2|349042|BEGIN GTID 0-2-6431| > |mysql-bin.29|349042|Table_map|2|349091|table_id: 19 (asic.changes)| > |mysql-bin.29|349091|Write_rows_v1|2|349127|table_id: 19 flags: > STMT_END_F| > |mysql-bin.29|349127|Table_map|2|349911|table_id: 24 (asic.tab_tracedata)| > |mysql-bin.29|349911|Write_rows_v1|2|350041|table_id: 24 flags: > STMT_END_F| > |mysql-bin.29|350041|Xid|2|350068|COMMIT /* xid=366 */| > == > triggered insert: > CREATE DEFINER=`root`@`%` TRIGGER `insertChanges` AFTER INSERT ON > `tab_tracedata` FOR EACH ROW BEGIN > INSERT INTO changes (changes.Header_RefID) SELECT NEW.Header_RefID; > END > events in eventlog: > |mysql-bin.29|343|Gtid|1|381|BEGIN GTID 0-1-6289| > |mysql-bin.29|381|Table_map|1|1165|table_id: 21 (asic.tab_tracedata)| > |mysql-bin.29|1165|Table_map|1|1214|table_id: 19 (asic.changes)| > |mysql-bin.29|1214|Write_rows_v1|1|3020|table_id: 21| > |mysql-bin.29|3020|Write_rows_v1|1|3071|table_id: 19 flags: STMT_END_F| > |mysql-bin.29|3071|Xid|1|3098|COMMIT /* xid=18 */| > > showcase: > CREATE TABLE test (id int auto_increment, data varchar(255),PRIMARY KEY (id)); > CREATE TABLE triggered (id int); > create trigger testtrigger AFTER insert ON test FOR EACH ROW INSERT INTO > triggered (triggered.id) SELECT NEW.id; > insert into test set data="lala"; > subscribe to eventlog via CaptureChangeMySQL module. There are 2 events > generated. Both inserts are mapped to the "triggered" table. thats wrong. one > should be mapped to the "test" table - the other one to the "triggered" table. > {"type":"insert","timestamp":1580393721000,"binlog_filename":"mysql-bin.29","binlog_position":5035809,"database":"nifi_test_case","table_name":"triggered","table_id":31,"columns":[\\{"id":1,"name":"id","column_type":4,"value":1} > ,\{"id":2,"value":null}]} > {"type":"insert","timestamp":1580393721000,"binlog_filename":"mysql-bin.29","binlog_position":5035849,"database":"nifi_test_case","table_name":"triggered","table_id":31,"columns":[\\{"id":1,"name":"id","column_type":4,"value":1} > ]}--=ixed 004F59E8C12584FF_= -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-8063) Add profile to Maven POM to enable NAR exclusion
[ https://issues.apache.org/jira/browse/NIFI-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-8063. Resolution: Won't Fix We have a new convention of adding profiles for individual capabilities, if there's a need for a bare-bones profile this case can be reopened, closing for now. > Add profile to Maven POM to enable NAR exclusion > > > Key: NIFI-8063 > URL: https://issues.apache.org/jira/browse/NIFI-8063 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Sometimes a bare-bones NiFi is all that is needed to test systems, > integrations, etc. It would be nice to be able to build a version of NiFi > without all the NARs in it (currently 1.5+ GB in total size). If this is done > as a profile in the assembly POM, the resulting artifacts can also be > dockerized (using the dockermaven module in the codebase). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12917) Content repository claim is being zero'd out
[ https://issues.apache.org/jira/browse/NIFI-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan F. resolved NIFI-12917. Resolution: Cannot Reproduce Fortunately/unfortunately I'm not able to reproduce the error in NiFi 1.25. I'll move to use this version on my servers. Thank you for your time and apologies for the ticket. > Content repository claim is being zero'd out > > > Key: NIFI-12917 > URL: https://issues.apache.org/jira/browse/NIFI-12917 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.19.1 > Environment: Red Hat 7 running Apache NiFi 1.19.1 with no custom > NARs. >Reporter: Evan F. >Priority: Critical > Attachments: example_MergeContent.txt > > > I've had rare but consistent instances of MergeContent processors merging > files and losing the content claim/payload while merging. I've pasted an > example log entry below where there's a size to the file pre-merge and the > post-merge content claim becomes zero. I've attached a text file with the > configuration of my MergeContent processor as well. This happens a fraction > of a percent of the time but it happens daily, somewhere around 10 times a > day. I scoured forums and previous tickets but I wasn't able to find a > relevant issue. I'm hoping someone can tell me if this has already been > addressed. > > INFO [Timer-Driven Process Thread-7] o.a.n.processors.standard.MergeContent > Merge Content[id=e6f460da-018d-1000-1c97-a6bd946d4f61] Merged > [StandardFlowFileRecord[uuid=0e5b7c30-021d-4bd9-9edb-1681c9c14d4e,claim=StandardContentClaim > [resourceClaim=StandardRe sourceClaim[id=1710769372188-3700202, > container=default, section=490], offset=0, > length=52430102],offset=0,name=testFile,size=52430102]] into > StandardFlowFileRecord[uuid=536bf8e4-915b-4dad-9e16-063b35e834c1,claim=,offset=0, > name=testFile.pkg,size=0]. Reason for merging: Maximum number of > entries reached -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-13069: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839090#comment-17839090 ] ASF subversion and git services commented on NIFI-13069: Commit fe5ef39ff2f342bd59c843b411b1fe4b1cf597ba in nifi's branch refs/heads/main from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fe5ef39ff2 ] NIFI-13069 Removed ConvertAvroToJSON Signed-off-by: Matt Burgess This closes #8675 > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.26.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13069 Removed ConvertAvroToJSON [nifi]
mattyb149 closed pull request #8675: NIFI-13069 Removed ConvertAvroToJSON URL: https://github.com/apache/nifi/pull/8675 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-13069: Fix Version/s: 1.26.0 > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.26.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839089#comment-17839089 ] ASF subversion and git services commented on NIFI-13069: Commit b066e1bf322244bd77dae8625c8bf540496d26dd in nifi's branch refs/heads/support/nifi-1.x from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b066e1bf32 ] NIFI-13069 Deprecated ConvertAvroToJSON Signed-off-by: Matt Burgess This closes #8672 > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13069 Deprecated ConvertAvroToJSON [nifi]
mattyb149 closed pull request #8672: NIFI-13069 Deprecated ConvertAvroToJSON URL: https://github.com/apache/nifi/pull/8672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13069 Deprecated ConvertAvroToJSON [nifi]
mattyb149 commented on PR #8672: URL: https://github.com/apache/nifi/pull/8672#issuecomment-2067112617 +1 LGTM, thanks for the improvement! Merging to support/nifi-1.x and adding to the Deprecated Components list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-4238) Error in QueryDatabaseTable (NiFi CDC support): Unable to execute SQL select query due to org.apache.nifi.processor.exception.ProcessException: Error during database quer
[ https://issues.apache.org/jira/browse/NIFI-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-4238. Resolution: Won't Fix Please feel free to reopen this case as necessary but since it is a driver limitation, hopefully it has been fixed in later versions of the driver. I'd really rather not hack the solution because non-standard JDBC types should be handled by the Database Type value in these processors. > Error in QueryDatabaseTable (NiFi CDC support): Unable to execute SQL select > query due to org.apache.nifi.processor.exception.ProcessException: Error > during database query or conversion of records to Avro > > > Key: NIFI-4238 > URL: https://issues.apache.org/jira/browse/NIFI-4238 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 > Environment: Centos >Reporter: Ella >Priority: Major > Attachments: Error.png, QueryDatabaseTableError.png, > config_file_1.png, config_file_2.png, config_file_3.png, diagram.png > > > Hi Guys, > I should retrieve only the new added records from the DB2 database to a file > by NiFi's CDC feature--QueryDatabaseTable processor; however, I have > encountered the Error during executing my dataflow scenario. I have > respectfully attached the snapshot of Error as well as the dataflow; I would > really appreciate if someone helped me after all. > Thanks a lot. > Sincerely, > Ella -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-12923. - Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839086#comment-17839086 ] ASF subversion and git services commented on NIFI-12923: Commit 3239f59746f7f2edf9375b60682bdd8ad7a6a7e9 in nifi's branch refs/heads/main from Balázs Gerner [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3239f59746 ] NIFI-12923 Added append avro mode to PutHDFS NIFI-12923 remove var keyword NIFI-12923 change property name NIFI-12923 Added property dependency for append_mode Signed-off-by: Matt Burgess This closes #8544 > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12923 Added append avro mode to PutHDFS [nifi]
mattyb149 closed pull request #8544: NIFI-12923 Added append avro mode to PutHDFS URL: https://github.com/apache/nifi/pull/8544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12923 Added append avro mode to PutHDFS [nifi]
mattyb149 commented on PR #8544: URL: https://github.com/apache/nifi/pull/8544#issuecomment-2067074203 +1 LGTM, verified the Avro append capability works as expected. Thanks for the improvement! Merging to main and support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12923) PutHDFS to support appending avro data
[ https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839080#comment-17839080 ] ASF subversion and git services commented on NIFI-12923: Commit 5462e69c96671954453e1a658795ac0f08a95012 in nifi's branch refs/heads/support/nifi-1.x from Balázs Gerner [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5462e69c96 ] NIFI-12923 Added append avro mode to PutHDFS NIFI-12923 remove var keyword NIFI-12923 change property name NIFI-12923 Added property dependency for append_mode Signed-off-by: Matt Burgess NIFI-12923: Fixed issues during backport to 1.x > PutHDFS to support appending avro data > -- > > Key: NIFI-12923 > URL: https://issues.apache.org/jira/browse/NIFI-12923 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > The goal of this ticket is to extend the PutHDFS processor with the ability > to append avro records. The processor already provides an option to set > 'append' as conflict resolution strategy, but that does not work correctly in > case of avro files, because the serialized avro file cannot be deserialized > again (because the binary content is invalid). > Some notes about the implementation: > * The user needs to explicitly select avro as file format and append as > conflict resolution mode to enable 'avro append' mode, otherwise regular > append mode will work just as before. There is no auto detection of mimetype > for the incoming flowfile. > * The records of the incoming flowfile and the ones in the existing avro > file need to conform to the same avro schema, otherwise the append operation > fails with incompatible schema. > * The 'avro append' mode should only work when compression type is set to > 'none', if any other compression type is selected in 'avro append' mode the > user should get a validation error. > The changes will have to be added to *support/nifi-1.x* branch also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13074) Reset lineage transform
Matt Gilman created NIFI-13074: -- Summary: Reset lineage transform Key: NIFI-13074 URL: https://issues.apache.org/jira/browse/NIFI-13074 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman When navigating from the lineage graph to the provenance event listing the transform needs to be reset otherwise the previous translation and zoom levels will be retained when the next graph is opened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-5519 Allow ListDatabaseTables to accept incoming connections and tweak GenerateTableFetch to work the same. [nifi]
dan-s1 commented on code in PR #8639: URL: https://github.com/apache/nifi/pull/8639#discussion_r1572734213 ## nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListDatabaseTables.java: ## @@ -270,24 +297,40 @@ public Set getRelationships() { return relationships; } +@OnScheduled +public void onScheduled(final ProcessContext context) { +if (!context.hasNonLoopConnection() && context.hasConnection(REL_FAILURE)) { +getLogger().error("The failure relationship can be used only if there is a non-loop incoming connection to this processor."); +} +if (!context.hasNonLoopConnection() && !context.getExecutionNode().equals(ExecutionNode.PRIMARY)) { +getLogger().error("If this processor has no incoming connection, then it should be run on Primary Node only."); +} +} + @Override public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException { +FlowFile fileToProcess = null; +if (context.hasIncomingConnection()) { +fileToProcess = session.get(); + +if (context.hasNonLoopConnection() && fileToProcess == null) { +// Incoming non-loop connection has no FlowFile available, do no work (see capability description) +// hasNonLoopConnection() protects from blocked execution when failure loops back to self and there is no other incoming connection. +return; +} +} + final ComponentLog logger = getLogger(); final DBCPService dbcpService = context.getProperty(DBCP_SERVICE).asControllerService(DBCPService.class); -final String catalog = context.getProperty(CATALOG).getValue(); -final String schemaPattern = context.getProperty(SCHEMA_PATTERN).getValue(); -final String tableNamePattern = context.getProperty(TABLE_NAME_PATTERN).getValue(); -final String[] tableTypes = context.getProperty(TABLE_TYPES).isSet() -? context.getProperty(TABLE_TYPES).getValue().split("\\s*,\\s*") -: null; -final boolean includeCount = context.getProperty(INCLUDE_COUNT).asBoolean(); -final long refreshInterval = context.getProperty(REFRESH_INTERVAL).asTimePeriod(TimeUnit.MILLISECONDS); +// Do not track refresh times if incoming flow file is non-null since FF causes relisting table names every time +final boolean isRefreshTracked = fileToProcess == null; +final long refreshInterval = !isRefreshTracked ? 0 : context.getProperty(REFRESH_INTERVAL).asTimePeriod(TimeUnit.MILLISECONDS); final StateMap stateMap; final Map stateMapProperties; try { -stateMap = session.getState(Scope.CLUSTER); -stateMapProperties = new HashMap<>(stateMap.toMap()); +stateMap = !isRefreshTracked ? null : session.getState(Scope.CLUSTER); +stateMapProperties = !isRefreshTracked ? null : new HashMap<>(stateMap.toMap()); Review Comment: ```suggestion stateMap = isRefreshTracked ? session.getState(Scope.CLUSTER) : null; stateMapProperties = isRefreshTracked ? new HashMap<>(stateMap.toMap()) : null; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13065: Adding initial bend points for self looping connections and connections that collide with existing connections [nifi]
rfellows commented on PR #8671: URL: https://github.com/apache/nifi/pull/8671#issuecomment-2067043767 reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572705711 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoHelper.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.mongodb; + +import java.util.EnumSet; +import org.apache.nifi.components.DescribedValue; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; + +public class PutMongoHelper { Review Comment: I do not see the value in this class. `ATTRIBUTE_MONGODB_UPDATE_MODE`, `updateModeMatches` and `MongoUpdateOption` I believe could pulled up to `AbstractMongoProcessor` since all of these are used by `PutMongo` and `PutMongoRecord`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572705711 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoHelper.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.mongodb; + +import java.util.EnumSet; +import org.apache.nifi.components.DescribedValue; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; + +public class PutMongoHelper { Review Comment: I do not see the value in this class. `ATTRIBUTE_MONGODB_UPDATE_MODE`, `updateModeMatches` and `MongoUpdateOption` I believe could pulled up to `AbstractMongoProcessor` since all of these are used by PutMongo and PutMongoRecord. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572705711 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoHelper.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.mongodb; + +import java.util.EnumSet; +import org.apache.nifi.components.DescribedValue; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; + +public class PutMongoHelper { Review Comment: I do not see the value in this class. `ATTRIBUTE_MONGODB_UPDATE_MODE`, `updateModeMatches` and `MongoUpdateOption` I believe could pulled up to `AbstractMongoProcessor`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572705711 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoHelper.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.mongodb; + +import java.util.EnumSet; +import org.apache.nifi.components.DescribedValue; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; + +public class PutMongoHelper { Review Comment: I do not see the value in this class. The `MongoUpdateOption` could be in its own file. `ATTRIBUTE_MONGODB_UPDATE_MODE` and `updateModeMatches` I believe could pulled up to `AbstractMongoProcessor`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572451044 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java: ## @@ -202,32 +225,49 @@ public void onTrigger(final ProcessContext context, final ProcessSession session session.read(flowFile, in -> StreamUtils.fillBuffer(in, content, true)); // parse -final Object doc = (mode.equals(MODE_INSERT) || (mode.equals(MODE_UPDATE) && updateMode.equals(UPDATE_WITH_DOC.getValue( +final Object doc = (processorMode.equals(MODE_INSERT) || (processorMode.equals(MODE_UPDATE) && flowfileType.equals(UPDATE_WITH_DOC.getValue( ? Document.parse(new String(content, charset)) : BasicDBObject.parse(new String(content, charset)); -if (MODE_INSERT.equalsIgnoreCase(mode)) { +if (MODE_INSERT.equalsIgnoreCase(processorMode)) { collection.insertOne((Document)doc); logger.info("inserted {} into MongoDB", new Object[] { flowFile }); } else { // update final boolean upsert = context.getProperty(UPSERT).asBoolean(); final String updateKey = context.getProperty(UPDATE_QUERY_KEY).evaluateAttributeExpressions(flowFile).getValue(); final String filterQuery = context.getProperty(UPDATE_QUERY).evaluateAttributeExpressions(flowFile).getValue(); -final Document query; +final Document updateQuery; -if (!StringUtils.isBlank(updateKey)) { -query = parseUpdateKey(updateKey, (Map)doc); +if (StringUtils.isNotBlank(updateKey)) { +updateQuery = parseUpdateKey(updateKey, (Map)doc); removeUpdateKeys(updateKey, (Map)doc); } else { -query = Document.parse(filterQuery); +updateQuery = Document.parse(filterQuery); } - -if (updateMode.equals(UPDATE_WITH_DOC.getValue())) { -collection.replaceOne(query, (Document)doc, new ReplaceOptions().upsert(upsert)); +UpdateResult updateResult; +if (flowfileType.equals(UPDATE_WITH_DOC.getValue())) { +updateResult = collection.replaceOne(updateQuery, (Document)doc, new ReplaceOptions().upsert(upsert)); } else { BasicDBObject update = (BasicDBObject)doc; update.remove(updateKey); -collection.updateOne(query, update, new UpdateOptions().upsert(upsert)); +UpdateOptions updateOptions = new UpdateOptions().upsert(upsert); +PropertyValue updateQueryMode = context.getProperty(MONGO_UPDATE_MODE); + +if (PutMongoHelper.updateModeMatches(MongoUpdateOption.UPDATE_ONE, updateQueryMode, flowFile)) { +updateResult = collection.updateOne(updateQuery, update, updateOptions); +} else if (PutMongoHelper.updateModeMatches(MongoUpdateOption.UPDATE_MANY, updateQueryMode, flowFile)) { +updateResult = collection.updateMany(updateQuery, update,updateOptions); +} else { +String flowfileUpdateMode = flowFile.getAttribute(PutMongoHelper.ATTRIBUTE_MONGODB_UPDATE_MODE); +throw new ProcessException("Unrecognized '" + PutMongoHelper.ATTRIBUTE_MONGODB_UPDATE_MODE + "' value '" + flowfileUpdateMode + "'"); +} +} +flowFile = session.putAttribute(flowFile, ATTRIBUTE_UPDATE_MATCH_COUNT, String.valueOf(updateResult.getMatchedCount())); Review Comment: ```suggestion } flowFile = session.putAttribute(flowFile, ATTRIBUTE_UPDATE_MATCH_COUNT, String.valueOf(updateResult.getMatchedCount())); ``` ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java: ## @@ -202,32 +225,49 @@ public void onTrigger(final ProcessContext context, final ProcessSession session session.read(flowFile, in -> StreamUtils.fillBuffer(in, content, true)); // parse -final Object doc = (mode.equals(MODE_INSERT) || (mode.equals(MODE_UPDATE) && updateMode.equals(UPDATE_WITH_DOC.getValue( +final Object doc = (processorMode.equals(MODE_INSERT) || (processorMode.equals(MODE_UPDATE) && flowfileType.equals(UPDATE_WITH_DOC.getValue( ? Document.parse(new String(content, charset)) : BasicDBObject.parse(new String(content, charset)); -if (MODE_INSERT.equalsIgnoreCase(mode)) { +if
[PR] NIFI-13054: Restoring Property Editor popups that provide EL Function documentation and Parameter descriptions [nifi]
mcgilman opened a new pull request, #8676: URL: https://github.com/apache/nifi/pull/8676 NIFI-13054: - Restoring Property Editor popups that provide EL Function documentation and Parameter descriptions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-13069: Status: Patch Available (was: In Progress) > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] Nifi 13069 [nifi]
dan-s1 opened a new pull request, #8675: URL: https://github.com/apache/nifi/pull/8675 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13062) Add support for read only details dialog
[ https://issues.apache.org/jira/browse/NIFI-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-13062: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Add support for read only details dialog > > > Key: NIFI-13062 > URL: https://issues.apache.org/jira/browse/NIFI-13062 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > When components are running we should provide a read only dialog to support > seeing configuration details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13062) Add support for read only details dialog
[ https://issues.apache.org/jira/browse/NIFI-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839048#comment-17839048 ] ASF subversion and git services commented on NIFI-13062: Commit 36c0ec4d7d2ad41b2ef2512402d102024d80dad1 in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=36c0ec4d7d ] NIFI-13062: Read only configuration dialogs (#8665) * NIFI-13062: - Updating Configuration dialogs to support a read only mode for Processors, Controller Services, Reporting Tasks, Ports, Connections, Process Groups, Remote Process Groups, Flow Analysis Rules, and Parameter Contexts. - For extensions points rendered the type in the header. * NIFI-13062: - Addressing review feedback. This closes #8665 > Add support for read only details dialog > > > Key: NIFI-13062 > URL: https://issues.apache.org/jira/browse/NIFI-13062 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > When components are running we should provide a read only dialog to support > seeing configuration details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows merged PR #8665: URL: https://github.com/apache/nifi/pull/8665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12125) nifi.security.autoreload.enabled does not trigger reload of SSLContext for cluster ServerSocket
[ https://issues.apache.org/jira/browse/NIFI-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12125: --- Assignee: David Handermann (was: Emilio Setiadarma) > nifi.security.autoreload.enabled does not trigger reload of SSLContext for > cluster ServerSocket > --- > > Key: NIFI-12125 > URL: https://issues.apache.org/jira/browse/NIFI-12125 > Project: Apache NiFi > Issue Type: Bug > Components: Security >Affects Versions: 1.23.2 > Environment: EKS Kube 1.27 >Reporter: Andrew Price >Assignee: David Handermann >Priority: Major > > Running Nifi 1.23 as a containerized cluster on EKS with > 'nifi.cluster.protocol.is.secure' set to true. Certificates are provisioned > to Nifi keystores using mounted Kube Secrets, periodically copied from secret > mount to keystore paths using sidecar container (avoiding NIFI-10425). > Upon certificate expiration and renewal (with > nifi.security.autoreload.enabled 'true'), Jetty server detects new > certificates and refreshes its SSLContext as expected. > However, cluster heartbeats (and potentially other cluster protocol messages) > begin to fail upon certificate expiration due to SSL validation failures. > Debugging of heartbeat events shows that nodes are sending heartbeats using > an updated SSL context, but cluster SocketProtocolListener and underlying > SocketListener continues to use stale SSLContext on an existing ServerSocket. > It may be required for the ServerSocket used by SocketProtocolListener to be > closed/recreated in order to refresh the underlying SSLContext, in order to > provide the same behaviour nifi.security.autoreload.enabled provides for the > HTTPS interface. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (NIFI-13054) Tooltip Improvements
[ https://issues.apache.org/jira/browse/NIFI-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reopened NIFI-13054: Reopening this Jira because the EL and Parameter description in the Property editors were broken in the previous commit. > Tooltip Improvements > > > Key: NIFI-13054 > URL: https://issues.apache.org/jira/browse/NIFI-13054 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Fix z-index issue with tooltips that hover over buttons. Currently the button > text is above the tooltip tip. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13035) Remove angular2jwt from Registry and Upgrade Node.js to 21 for nifi-web-ui build
[ https://issues.apache.org/jira/browse/NIFI-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13035: Summary: Remove angular2jwt from Registry and Upgrade Node.js to 21 for nifi-web-ui build (was: UI - remove angular2jwt from Nifi Registry) > Remove angular2jwt from Registry and Upgrade Node.js to 21 for nifi-web-ui > build > > > Key: NIFI-13035 > URL: https://issues.apache.org/jira/browse/NIFI-13035 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13073 Change flow version from int to String to allow more flexi… [nifi]
bbende opened a new pull request, #8674: URL: https://github.com/apache/nifi/pull/8674 …bility for custom registry clients # Summary [NIFI-13073](https://issues.apache.org/jira/browse/NIFI-13073) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-3714) Encrypt config tool replaces wrong key/value pair with encrypted value and exposes plain sensitive value
[ https://issues.apache.org/jira/browse/NIFI-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-3714. Fix Version/s: 2.0.0-M1 Assignee: David Handermann (was: Andy LoPresto) Resolution: Fixed > Encrypt config tool replaces wrong key/value pair with encrypted value and > exposes plain sensitive value > > > Key: NIFI-3714 > URL: https://issues.apache.org/jira/browse/NIFI-3714 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.1.1 >Reporter: Andy LoPresto >Assignee: David Handermann >Priority: Major > Labels: config, encryption, security > Fix For: 2.0.0-M1 > > > In testing the encrypted provenance repository key management system, I found > an issue where the {{$ ./bin/encrypt-config.sh}} command can encrypt the > correct sensitive value but replace the wrong key/value pair in the encrypted > {{nifi.properties}} file. > Example: > *Existing nifi.properties* > {code} > nifi.provenance.repository.implementation=org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository > nifi.provenance.repository.debug.frequency=1_000_000 > nifi.provenance.repository.encryption.key.provider.implementation=org.apache.nifi.provenance.StaticKeyProvider > nifi.provenance.repository.encryption.key.provider.location= > nifi.provenance.repository.encryption.key.id=Key1 > nifi.provenance.repository.encryption.key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > nifi.sensitive.props.additional.keys=nifi.provenance.repository.encryption.key > {code} > *After running {{./bin/encrypt-config.sh -b conf/bootstrap.conf -n > conf/nifi.properties -k > }}* > {code} > # Provenance Repository Properties > nifi.provenance.repository.implementation=org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository > nifi.provenance.repository.debug.frequency=1_000_000 > nifi.provenance.repository.encryption.key=aYDkDKys1ENr3gp+||sTBPpMlIvHcOLTGZlfWct8r9RY8BuDlDkoaYmGJ/9m9af9tZIVzcnDwvYQAaIKxRGF7vI2yrY7Xd6x9GTDnWGiGiRXlaP458BBMMgfzH2O8 > nifi.provenance.repository.encryption.key.protected=aes/gcm/256 > nifi.provenance.repository.encryption.key.provider.location= > nifi.provenance.repository.encryption.key.id=Key1 > nifi.provenance.repository.encryption.key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > nifi.sensitive.props.additional.keys=nifi.provenance.repository.encryption.key > {code} > *When it should be* > {code} > # Provenance Repository Properties > nifi.provenance.repository.implementation=org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository > nifi.provenance.repository.debug.frequency=1_000_000 > nifi.provenance.repository.encryption.key.provider.implementation=org.apache.nifi.provenance.StaticKeyProvider > nifi.provenance.repository.encryption.key.provider.location= > nifi.provenance.repository.encryption.key.id=Key1 > nifi.provenance.repository.encryption.key=aYDkDKys1ENr3gp+||sTBPpMlIvHcOLTGZlfWct8r9RY8BuDlDkoaYmGJ/9m9af9tZIVzcnDwvYQAaIKxRGF7vI2yrY7Xd6x9GTDnWGiGiRXlaP458BBMMgfzH2O8 > nifi.provenance.repository.encryption.key.protected=aes/gcm/256 > nifi.sensitive.props.additional.keys=nifi.provenance.repository.encryption.key > {code} > I verified (using a test fixture) that the cipher text is the correct > encryption of > {{0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210}} with a > random IV (Base64-encoded and embedded in the cipher text as > {{aYDkDKys1ENr3gp+}} and the key > {{}}. It does > not appear to be an issue with the encryption but with the replacement logic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13073) Update FlowRegistryClient API to use a String for version instead of int
Bryan Bende created NIFI-13073: -- Summary: Update FlowRegistryClient API to use a String for version instead of int Key: NIFI-13073 URL: https://issues.apache.org/jira/browse/NIFI-13073 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende The current FlowRegistryClient API is heavily based on extracting an API from what originally existed from the initial NiFi Registry integration. In NiFi Registry, each version of a flow gets assigned a one up integer number and that is the "version", so as a result the API on NiFi side uses an `int` for all references to a version. This makes it difficult to implement a FlowRegistryClient for another system that doesn't use a one up numbering schema, for example in Git the version would be considered the commit hash which would be a String. The proposal here is to switch from `int version` to `String version` in the FlowRegistryClient interface and anything above that on NiFi side. The implementation can then decide how to interpret the String, meaning the NiFi Registry implementation can simply do `Integer.parseInt(version)` if it needs to determine the next version by performing `currentVersion + `. There is also some code that expects to sort a list of version metadata results by comparing the the version fields. This should be changed to sort based on the timestamp of each version. This will technically be a breaking change to the FlowRegistryClient interface, but should be very straight forward for anyone to update, and it is targeted for a 2.0.0 release where API changes can happen. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12317) REST API Rejects Keycloak OIDC Access Tokens with HTTP 401
[ https://issues.apache.org/jira/browse/NIFI-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12317: Affects Version/s: (was: 1.23.2) Status: Patch Available (was: Open) > REST API Rejects Keycloak OIDC Access Tokens with HTTP 401 > -- > > Key: NIFI-12317 > URL: https://issues.apache.org/jira/browse/NIFI-12317 > Project: Apache NiFi > Issue Type: Bug > Components: Security >Reporter: Anmol >Assignee: David Handermann >Priority: Major > > I have integrated Nifi with keycloak and NIfi UI is working fine. But if i am > hitting nifi-api with keycloak access token , nifi is giving 401 error. > > error: Unauthorized error="invalid_token", error_description="An error > occurred while attempting to decode the Jwt: Signed JWT rejected: Another > algorithm expected, or no matching key(s) found", > error_uri="https://tools.ietf.org/html/rfc6750#section-3.1; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12317) REST API Rejects Keycloak OIDC Access Tokens with HTTP 401
[ https://issues.apache.org/jira/browse/NIFI-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12317: --- Assignee: David Handermann > REST API Rejects Keycloak OIDC Access Tokens with HTTP 401 > -- > > Key: NIFI-12317 > URL: https://issues.apache.org/jira/browse/NIFI-12317 > Project: Apache NiFi > Issue Type: Bug > Components: Security >Affects Versions: 1.23.2 >Reporter: Anmol >Assignee: David Handermann >Priority: Major > > I have integrated Nifi with keycloak and NIfi UI is working fine. But if i am > hitting nifi-api with keycloak access token , nifi is giving 401 error. > > error: Unauthorized error="invalid_token", error_description="An error > occurred while attempting to decode the Jwt: Signed JWT rejected: Another > algorithm expected, or no matching key(s) found", > error_uri="https://tools.ietf.org/html/rfc6750#section-3.1; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12317) REST API Rejects Keycloak OIDC Access Tokens with HTTP 401
[ https://issues.apache.org/jira/browse/NIFI-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12317: Issue Type: Improvement (was: Bug) > REST API Rejects Keycloak OIDC Access Tokens with HTTP 401 > -- > > Key: NIFI-12317 > URL: https://issues.apache.org/jira/browse/NIFI-12317 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Reporter: Anmol >Assignee: David Handermann >Priority: Major > > I have integrated Nifi with keycloak and NIfi UI is working fine. But if i am > hitting nifi-api with keycloak access token , nifi is giving 401 error. > > error: Unauthorized error="invalid_token", error_description="An error > occurred while attempting to decode the Jwt: Signed JWT rejected: Another > algorithm expected, or no matching key(s) found", > error_uri="https://tools.ietf.org/html/rfc6750#section-3.1; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5582) Integrate legacy behavior of HashAttribute into CryptographicHashAttribute
[ https://issues.apache.org/jira/browse/NIFI-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-5582. Resolution: Won't Fix Hash Attribute Processors have been removed from the main branch for NiFi 2.0 in favor of hashing features incorporated in Expression Language, available through UpdateAttribute. > Integrate legacy behavior of HashAttribute into CryptographicHashAttribute > -- > > Key: NIFI-5582 > URL: https://issues.apache.org/jira/browse/NIFI-5582 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: attribute, cryptography, hash > > There has been discussion on the mailing lists regarding the use of the > existing {{HashAttribute}} processor and the introduction of > {{CryptographicHashAttribute}}. The behavior of these processors does not > currently overlap, but {{CHA}} can be made to include {{HA}}'s functionality > in the documented use cases (to generate a unique identifier over a set of > specific attributes and values), if not its exact behavior. > *From discussion* > {quote} > Given your well-described use cases for HA, I think I may be able to provide > that in CHA as well. I would expect to add a dropdown PD for “attribute > enumeration style” and offer “individual” (each hash is generated on a single > attribute), “list” (each hash is generated over an ordered, delimited list of > literal matches), and “regex” (each hash is generated over an ordered list of > all attribute names matching the provided regex). Then the dynamic properties > would describe the output, as happens in the existing PR. Maybe a custom > delimiter property is needed too, but for now ‘’ could be used to join the > values. I’ll write up a Jira for this, and hopefully you can both let me know > if this meets your requirements. > Example: > *Incoming Flowfile* > attributes: [username: “alopresto”, role: “security”, email: > “alopre...@apache.org”, git_account: “alopresto”] > *CHA Properties (Individual)* > attribute_enumeration_style: “individual” > (dynamic) username_sha256: “username” > (dynamic) git_account_sha256: “git_account” > *Behavior (Individual)* > username_sha256 = git_account_sha256 = $(echo -n "alopresto" | shasum -a 256) > = 600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23 > *Resulting Flowfile (Individual)* > > attributes: [username: “alopresto”, role: “security”, email: > “alopre...@apache.org”, git_account: “alopresto”, username_sha256: > “600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23”, > git_account_sha256: > “600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23"] > *CHA Properties (List)* > attribute_enumeration_style: “list” > (dynamic) username_and_email_sha256: “username, email” > (dynamic) git_account_sha256: “git_account” > *Behavior (List)* > username_and_email_sha256 = $(echo -n "aloprestoalopre...@apache.org" | > shasum -a 256) = > 22a11b7b3173f95c23a1f434949ec2a2e66455b9cb26b7ebc90afca25d91333f > git_account_sha256 = $(echo -n "alopresto" | shasum -a 256) = > 600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23 > *Resulting Flowfile (List)* > > attributes: [username: “alopresto”, role: “security”, email: > “alopre...@apache.org”, git_account: “alopresto”, username_email_sha256: “ > 22a11b7b3173f95c23a1f434949ec2a2e66455b9cb26b7ebc90afca25d91333f”, > git_account_sha256: > “600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23”] > *CHA Properties (Regex)* > attribute_enumeration_style: “regex” > (dynamic) all_sha256: “.*” > (dynamic) git_account_sha256: “git_account” > *Behavior (Regex)* > all_sha256 = sort(attributes_that_match_regex) = [email, git_account, role, > username] = $(echo -n "alopresto@apache.orgaloprestosecurityalopresto" | > shasum -a 256) = > b370fdf0132933cea76e3daa3d4a437bb8c571dd0cd0e79ee5d7759cf64efced > git_account_sha256 = $(echo -n "alopresto" | shasum -a 256) = > 600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23 > *Resulting Flowfile (Regex)* > > attributes: [username: “alopresto”, role: “security”, email: > “alopre...@apache.org”, git_account: “alopresto”, all_sha256: “ > b370fdf0132933cea76e3daa3d4a437bb8c571dd0cd0e79ee5d7759cf64efced”, > git_account_sha256: > “600973dc8f2b7bb2a20651ebefe4bf91c5295afef19f4d5b9994d581f5a68a23”] > {quote} > This will necessitate switching the "order" of dynamic properties in > {{CryptographicHashAttribute}} -- rather than a dynamic property > *existing_attribute_name* - {{new_attribute_name_containing_hash}}, the > ordering will be *new_attribute_name_containing_hash* - >
[jira] [Commented] (NIFI-12018) Allow return values from UpdateRecord to be non-String values
[ https://issues.apache.org/jira/browse/NIFI-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839030#comment-17839030 ] Steven Youtsey commented on NIFI-12018: --- When the value of field.value is null, it would be helpful if that would be retained. Currently, it is converted (silently) to an empty string. For example, /someField => ${field.value} someField should remain null > Allow return values from UpdateRecord to be non-String values > - > > Key: NIFI-12018 > URL: https://issues.apache.org/jira/browse/NIFI-12018 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > In the UpdateRecord processor, it allows using Expression Language to refer > to `field.value` and `field.name`. Often, these are used for things like date > manipulation, etc. but the return value is always a String because it's > evaluating Expression Language. > Under the hood, Expression Language's {{Query}} object exposes a > {{{}ResultType{}}}. We should add two methods to the {{PropertyValue}} object: > {{ResultType getResultType()}} > {{Object asObject()}} > Where {{getResultType()}} will return {{ResultType.STRING}} if no Expression > Language is present / has been evaluated, and the {{asObject()}} method will > return one of the following types of values based on the {{{}ResultType{}}}: > {code:java} > ResultType | Java Type > STRING | String > BOOLEAN | Boolean > WHOLE_NUMBER | Long > DATE | Date > INSTANT | java.time.Instant > DECIMAL | Double > NUMBER | Number{code} > We should then update {{UpdateRecord}} to make use of {{asObject()}} instead > of {{{}getValue(){}}}. This would allow us to create expressions such as > {code:java} > ${field.value:toDate('/MM/dd HH:mm:ss'):toNumber()} {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13069 Deprecated ConvertAvroToJSON [nifi]
exceptionfactory commented on code in PR #8672: URL: https://github.com/apache/nifi/pull/8672#discussion_r1572537399 ## nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/main/java/org/apache/nifi/processors/avro/ConvertAvroToJSON.java: ## @@ -65,6 +66,9 @@ + "multiple Avro records, the resultant FlowFile will contain a JSON Array containing all of the Avro records or a sequence of JSON Objects. If an incoming FlowFile does " + "not contain any records, an empty JSON object is the output. Empty/Single Avro record FlowFile inputs are optionally wrapped in a container as dictated by 'Wrap Single Record'") @WritesAttribute(attribute = "mime.type", description = "Sets the mime type to application/json") +@DeprecationNotice(reason = "ConvertAvroToJSON is no longer needed since there is the AvroReader which along with JsonRecordSetWriter" + Review Comment: The general expectation for all deprecated components is removal in future versions, so it doesn't seem necessary to state that in particular. Once this is merged, it should be listed in the [Deprecated Components and Features](https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12550) Support OIDC Device Authorization Grant for API
[ https://issues.apache.org/jira/browse/NIFI-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12550: Affects Version/s: (was: 1.23.2) Status: Patch Available (was: In Progress) > Support OIDC Device Authorization Grant for API > --- > > Key: NIFI-12550 > URL: https://issues.apache.org/jira/browse/NIFI-12550 > Project: Apache NiFi > Issue Type: Improvement > Components: Security > Environment: NiFi with OIDC provider configured >Reporter: Igor Milavec >Assignee: David Handermann >Priority: Major > > Please add support for OIDC Device Authorization Grant. This is useful for > running scripts that access the NiFi API from the CLI. At this time the > options are: > # Copy __Secure-Authorization-Bearer cookie from the browser session: not > really a good practice, work and error prone > # Enable MTLS: painful for the users as the browser starts to frequently > challenge for the client cert and even if it worked fine, client certificate > management process is typically lagging behind OIDC identity management > # Use passwords: insecure and prohibited by policy > Having an API endpoint in the Access group that would allow the caller to > exchange OIDC id or refresh token for a NiFi session token would be perfect > for this use case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12858 Correct Order of Previous Property Values [nifi]
exceptionfactory opened a new pull request, #8673: URL: https://github.com/apache/nifi/pull/8673 # Summary [NIFI-12858](https://issues.apache.org/jira/browse/NIFI-12858) Corrects the ordering of previous property values shown when hovering over the information icon for component configuration. Migration from H2 to Xodus in NiFi 1.24.0 and following inadvertently changed the order to return the newest entries first, instead of the oldest entries first. The correction reverses the default ordering and updated unit tests confirm the expected behavior. This change should be backported to the support branch. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12858: Labels: backport-needed (was: ) > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.25.0 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 10m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12858: Affects Version/s: 2.0.0-M2 1.24.0 2.0.0-M1 > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1, 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 10m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12858: Status: Patch Available (was: In Progress) > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.25.0 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 10m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13069 Deprecated ConvertAvroToJSON [nifi]
mattyb149 commented on code in PR #8672: URL: https://github.com/apache/nifi/pull/8672#discussion_r1572529175 ## nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/main/java/org/apache/nifi/processors/avro/ConvertAvroToJSON.java: ## @@ -65,6 +66,9 @@ + "multiple Avro records, the resultant FlowFile will contain a JSON Array containing all of the Avro records or a sequence of JSON Objects. If an incoming FlowFile does " + "not contain any records, an empty JSON object is the output. Empty/Single Avro record FlowFile inputs are optionally wrapped in a container as dictated by 'Wrap Single Record'") @WritesAttribute(attribute = "mime.type", description = "Sets the mime type to application/json") +@DeprecationNotice(reason = "ConvertAvroToJSON is no longer needed since there is the AvroReader which along with JsonRecordSetWriter" + Review Comment: Maybe mention that it is no longer maintained and is subject to removal in a future release (unless the generated doc does that automatically) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-13069 Deprecated ConvertAvroToJSON [nifi]
dan-s1 opened a new pull request, #8672: URL: https://github.com/apache/nifi/pull/8672 # Summary [NIFI-13069](https://issues.apache.org/jira/browse/NIFI-13069) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-13069) Remove ConvertAvroToJSON
[ https://issues.apache.org/jira/browse/NIFI-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839025#comment-17839025 ] ASF subversion and git services commented on NIFI-13069: Commit 55907c0e55c427b5b6fd50b71c0615f968d07aea in nifi's branch refs/heads/NIFI-13069 from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=55907c0e55 ] NIFI-13069 Deprecated ConvertAvroToJSON > Remove ConvertAvroToJSON > > > Key: NIFI-13069 > URL: https://issues.apache.org/jira/browse/NIFI-13069 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > > The ConvertRecord configured with an AvroReader and a JsonRecordSetWriter > accomplishes the same as this processor. Per the following [thread > |https://lists.apache.org/thread/5cvvfn8oq0ttcxz0pggs8wr4xn3608tq] it was > agreed this should be deprecated in 1.x and removed for 2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) *** Support Processor Details dialog *** Support Connection Details dialog * Counters (NIFI-12415) ** Counter table has extra unnecessary can modify check (NIFI-12948) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Node table (Disconnect/Connect/Load Balance/Etc) (NIFI-13053) ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional (NIFI-12948) * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none (NIFI-12948) * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default (NIFI-12948) * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * Replay last event (NIFI-12445) * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering (NIFI-13005) * Change Component Version (NIFI-13034) * Consider PG permissions in Toolbox (NIFI-12683) * PG Version (NIFI-12963 & NIFI-12995) ** Start (NIFI-12963) ** Commit (NIFI-12963) ** Force Commit (NIFI-12963) ** Show changes (NIFI-12995) ** Revert changes (NIFI-12995) ** Change Flow version (NIFI-12995) ** Stop (NIFI-12963) * Configure PG (NIFI-12417) * Configure Label * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Status Bar ** Stop and Configure ** Open Custom UI (NIFI-12958) ** Property History (NIFI-13047) ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * Terminate Threads (context menu) (NIFI-13068) * Enable/Disable (context menu/operate panel) * More Details (Processor, Controller Service, Reporting Task) (NIFI-13062) * Download Flow (NIFI-13029) * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color (context menu/operate panel) * Move to Front (NIFI-13044) * Copy/Paste (context menu/operate panel) (NIFI-13059) * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build (NIFI-12941) * Update all API calls to consider disconnect node confirmation (NIFI-13001) * Update API calls to use uiOnly flag (NIFI-12950) * Use polling interval from API * Load FlowConfiguration in guard (NIFI-12948) * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Parameter ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users
[jira] [Updated] (NIFI-13068) Add support to Terminate active threads
[ https://issues.apache.org/jira/browse/NIFI-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13068: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add support to Terminate active threads > --- > > Key: NIFI-13068 > URL: https://issues.apache.org/jira/browse/NIFI-13068 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Add the ability to terminate threads from the context menu for a component > that is stopped but still has active threads. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13068: Adding support to terminate threads for Processors that are no longer running [nifi]
scottyaslan merged PR #8668: URL: https://github.com/apache/nifi/pull/8668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-13068) Add support to Terminate active threads
[ https://issues.apache.org/jira/browse/NIFI-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839015#comment-17839015 ] ASF subversion and git services commented on NIFI-13068: Commit 8c3c1eea31afb4a3421bdf074e9ca38d2128fb0b in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8c3c1eea31 ] NIFI-13068: (#8668) - Adding support to terminate threads for Processors that are no longer running. - Updating import for concatLatestFrom. This closes #8668 > Add support to Terminate active threads > --- > > Key: NIFI-13068 > URL: https://issues.apache.org/jira/browse/NIFI-13068 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Add the ability to terminate threads from the context menu for a component > that is stopped but still has active threads. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] MINIFICPP-2334 win: smb and loki extensions on by default [nifi-minifi-cpp]
szaszm opened a new pull request, #1764: URL: https://github.com/apache/nifi-minifi-cpp/pull/1764 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-2334) win: smb and loki extensions on by default
Marton Szasz created MINIFICPP-2334: --- Summary: win: smb and loki extensions on by default Key: MINIFICPP-2334 URL: https://issues.apache.org/jira/browse/MINIFICPP-2334 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Marton Szasz -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12224: Add support for updateMany operation with operators enabled option in `PutMongo` processor [nifi]
dan-s1 commented on code in PR #8610: URL: https://github.com/apache/nifi/pull/8610#discussion_r1572407325 ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoRecord.java: ## @@ -141,8 +135,8 @@ public class PutMongoRecord extends AbstractMongoProcessor { .displayName("Update Mode") .dependsOn(UPDATE_KEY_FIELDS) .description("Choose between updating a single document or multiple documents per incoming record.") -.allowableValues(UPDATE_ONE, UPDATE_MANY, UPDATE_FF_ATTRIBUTE) -.defaultValue(UPDATE_ONE.getValue()) +.allowableValues(MongoUpdateOption.allowedValues()) Review Comment: ```suggestion .allowableValues(MongoUpdateOption.class) ``` ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoHelper.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.mongodb; + +import java.util.EnumSet; +import org.apache.nifi.components.DescribedValue; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; + +public class PutMongoHelper { + + public static final String ATTRIBUTE_MONGODB_UPDATE_MODE = "mongodb.update.mode"; + + /** + * Checks if given update mode option matches for the incoming flow file + * @param updateModeToMatch the value against which processor's mode is compared + * @param processorMode the value coming from running processor + * @param flowFile incoming flow file to extract processor mode + * @return true if the incoming files update mode matches with updateModeToMatch + */ + public static boolean updateModeMatches(MongoUpdateOption updateModeToMatch, PropertyValue processorMode, FlowFile flowFile) { +String updateMode = processorMode.getValue(); + +return updateModeToMatch.matches(updateMode) || (MongoUpdateOption.UPDATE_FF_ATTRIBUTE.matches(updateMode) && updateModeToMatch.getValue() + .equalsIgnoreCase(flowFile.getAttribute(PutMongoHelper.ATTRIBUTE_MONGODB_UPDATE_MODE))); + } + + public enum MongoUpdateOption implements DescribedValue { +UPDATE_ONE("one", "Update One", "Updates only the first document that matches the query."), +UPDATE_MANY("many", "Update Many", "Updates every document that matches the query."), +UPDATE_FF_ATTRIBUTE("flowfile-attribute", "Use '" + ATTRIBUTE_MONGODB_UPDATE_MODE + "' flowfile attribute.", +"Use the value of the '" + ATTRIBUTE_MONGODB_UPDATE_MODE + "' attribute of the incoming flowfile. Acceptable values are 'one' and 'many'."); +private final String value; +private final String displayName; +private final String description; + +MongoUpdateOption(final String value, final String displayName, final String description) { + this.value = value; + this.displayName = displayName; + this.description = description; +} + +public static EnumSet allowedValues() { + return EnumSet.of( + UPDATE_ONE, + UPDATE_MANY, + UPDATE_FF_ATTRIBUTE + ); +} + Review Comment: ```suggestion ``` This method is not necessary as `.allowableValues` in `PropertyDescriptor.Builder` takes a class e.g. `.allowableValues(MongoUpdateOption.class)` ## nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongo.java: ## @@ -82,39 +96,51 @@ public class PutMongo extends AbstractMongoProcessor { .description("When true, inserts a document if no document matches the update query criteria; this property is valid only when using update mode, " + "otherwise it is ignored") .required(true) +.dependsOn(MODE,MODE_UPDATE) .allowableValues("true", "false") .addValidator(StandardValidators.BOOLEAN_VALIDATOR) .defaultValue("false") .build(); static final PropertyDescriptor UPDATE_QUERY_KEY = new PropertyDescriptor.Builder() .name("Update Query Key") -.description("Key name used to build
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
mcgilman commented on PR #8665: URL: https://github.com/apache/nifi/pull/8665#issuecomment-2066633138 > Parameter Providers... do we need to support a read only mode from the Controller Settings? If not, we should remove the More Info icon in the table. Yes we should support a read only mode for Parameter Providers. It should be driven by the users `canWrite` permission. I missed this on my initial pass. Will udpate... > Are we opening these dialogs from Summary screen more info icons as part of another effort? That is not intended to be part of this effort. That will be addressed in a follow up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572404410 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/settings/ui/reporting-tasks/edit-reporting-task/edit-reporting-task.component.ts: ## @@ -137,13 +144,16 @@ export class EditReportingTask { // build the form this.editReportingTaskForm = this.formBuilder.group({ name: new FormControl(request.reportingTask.component.name, Validators.required), -state: new FormControl(request.reportingTask.component.state === 'STOPPED', Validators.required), +state: new FormControl( +{ value: request.reportingTask.component.state !== 'DISABLED', disabled: this.readonly }, Review Comment: Thanks for the clarification. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
mcgilman commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572400114 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/settings/ui/parameter-providers/edit-parameter-provider/edit-parameter-provider.component.scss: ## @@ -17,6 +17,10 @@ @use '@angular/material' as mat; +h2.mdc-dialog__title::before { +height: 0; +} Review Comment: That is overriding the Angular default style that triggers when the `mdc-dialog__title` contains additional content. If I recall correctly `:before` is assigned 40px. In cases where we're rendering additional component details in the header directly it was causing unwanted additional empty space at the top of the dialog. Happy to consider alternatives if you have suggestions. Or maybe we'll make additional changes when we introduce the `status` bar within these dialogs and add support for actions like Stop and Configure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
mcgilman commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572394232 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/settings/ui/reporting-tasks/edit-reporting-task/edit-reporting-task.component.ts: ## @@ -137,13 +144,16 @@ export class EditReportingTask { // build the form this.editReportingTaskForm = this.formBuilder.group({ name: new FormControl(request.reportingTask.component.name, Validators.required), -state: new FormControl(request.reportingTask.component.state === 'STOPPED', Validators.required), +state: new FormControl( +{ value: request.reportingTask.component.state !== 'DISABLED', disabled: this.readonly }, Review Comment: - `readonly` is only supported by `input` and `textarea`. These are always set in the template. - If the field we're attempting to disable is in the form directly (has the formControlName attribute) and you attempt to disable in the template, Angular will provide the following warning. This happens for components like a checkbox or any `ControlValueAccessor`. It is recommended that you enable and disable through the form API. > It looks like you're using the disabled attribute with a reactive form directive. If you set disabled to true > when you set up this control in your component class, the disabled attribute will actually be set in the DOM for > you. We recommend using this approach to avoid 'changed after checked' errors. > > Example: > // Specify the `disabled` property at control creation time: > form = new FormGroup({ > first: new FormControl({value: 'Nancy', disabled: true}, Validators.required), > last: new FormControl('Drew', Validators.required) > }); > > // Controls can also be enabled/disabled after creation: > form.get('first')?.enable(); > form.get('last')?.disable(); - If the field your attempting to disable is not directly a form control (does not have the formControlName attribute - like an option in a mat-select) we disable through the template. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13065) Self looping connection missing bend points
[ https://issues.apache.org/jira/browse/NIFI-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13065: --- Status: Patch Available (was: In Progress) > Self looping connection missing bend points > --- > > Key: NIFI-13065 > URL: https://issues.apache.org/jira/browse/NIFI-13065 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When adding a self looping connection the bend points are missing. Also, when > connecting a source and destination that already have a connection, we are no > longer inserting and shifting bend points to avoid overlapping. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13065: Adding initial bend points for self looping connections and connections that collide with existing connections [nifi]
mcgilman opened a new pull request, #8671: URL: https://github.com/apache/nifi/pull/8671 NIFI-13065: - Adding initial bend points for self looping connections and connections that collide with existing connections. - Merging two actions into one for opening the new connection dialog. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572366502 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/settings/ui/reporting-tasks/edit-reporting-task/edit-reporting-task.component.ts: ## @@ -137,13 +144,16 @@ export class EditReportingTask { // build the form this.editReportingTaskForm = this.formBuilder.group({ name: new FormControl(request.reportingTask.component.name, Validators.required), -state: new FormControl(request.reportingTask.component.state === 'STOPPED', Validators.required), +state: new FormControl( +{ value: request.reportingTask.component.state !== 'DISABLED', disabled: this.readonly }, Review Comment: Why are some controls set to readonly in the template and some in the form group? What drives the decision as to how we should disable a control? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572361984 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/settings/ui/parameter-providers/edit-parameter-provider/edit-parameter-provider.component.scss: ## @@ -17,6 +17,10 @@ @use '@angular/material' as mat; +h2.mdc-dialog__title::before { +height: 0; +} Review Comment: Why do we need this for the edit dialogs but not for any of the others? Is it just because we have divs in the h2? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-2333) Don't register example python processors
Marton Szasz created MINIFICPP-2333: --- Summary: Don't register example python processors Key: MINIFICPP-2333 URL: https://issues.apache.org/jira/browse/MINIFICPP-2333 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Marton Szasz -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows commented on code in PR #8665: URL: https://github.com/apache/nifi/pull/8665#discussion_r1572327397 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/service/canvas-context-menu.service.ts: ## @@ -477,14 +477,25 @@ export class CanvasContextMenu implements ContextMenuDefinitionProvider { } }, { -condition: (selection: any) => { -// TODO - hasDetails -return false; +condition: (selection: d3.Selection) => { +return this.canvasUtils.hasDetails(selection); Review Comment: The `hasDetails` logic needs updated to support a Process Group that a user does not have modify rights to. Currently the user can't "View Configuration" in this scenario. This is most likely just due to the fact that we separated Configure and Controller Services and the Configure option was the only option and visible for a read-only PG. _Also_, once I updated the logic locally... I noticed that the order of the menu options was off. We show `Configure`, `Controller Services`. But when in read only mode, it is shown as `Controller Services`, `View Configuration`. We should move `View Configuration` before `Controller Services`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13062: Read only configuration dialogs [nifi]
rfellows commented on PR #8665: URL: https://github.com/apache/nifi/pull/8665#issuecomment-2066434378 reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13070) Upgrade Netty to 4.1.109
[ https://issues.apache.org/jira/browse/NIFI-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13070: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Netty to 4.1.109 > > > Key: NIFI-13070 > URL: https://issues.apache.org/jira/browse/NIFI-13070 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to > [4.1.109|https://netty.io/news/2024/04/15/4-1-109-Final.html] to incorporate > several bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13070) Upgrade Netty to 4.1.109
[ https://issues.apache.org/jira/browse/NIFI-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838970#comment-17838970 ] ASF subversion and git services commented on NIFI-13070: Commit 74e27868e342c2078b29506faf0ebbd050b25b39 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=74e27868e3 ] NIFI-13070 Upgraded Netty from 4.1.108 to 4.1.109 Signed-off-by: Pierre Villard This closes #8667. > Upgrade Netty to 4.1.109 > > > Key: NIFI-13070 > URL: https://issues.apache.org/jira/browse/NIFI-13070 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 20m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to > [4.1.109|https://netty.io/news/2024/04/15/4-1-109-Final.html] to incorporate > several bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13070) Upgrade Netty to 4.1.109
[ https://issues.apache.org/jira/browse/NIFI-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838969#comment-17838969 ] ASF subversion and git services commented on NIFI-13070: Commit b81203aac2d945b449849a368fd601c140a93673 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b81203aac2 ] NIFI-13070 Upgraded Netty from 4.1.108 to 4.1.109 Signed-off-by: Pierre Villard This closes #8667. > Upgrade Netty to 4.1.109 > > > Key: NIFI-13070 > URL: https://issues.apache.org/jira/browse/NIFI-13070 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 10m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to > [4.1.109|https://netty.io/news/2024/04/15/4-1-109-Final.html] to incorporate > several bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13070 Upgrade Netty from 4.1.108 to 4.1.109 [nifi]
asfgit closed pull request #8667: NIFI-13070 Upgrade Netty from 4.1.108 to 4.1.109 URL: https://github.com/apache/nifi/pull/8667 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-13067) Upgrade Spring Security to 6.2.4
[ https://issues.apache.org/jira/browse/NIFI-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838968#comment-17838968 ] ASF subversion and git services commented on NIFI-13067: Commit 6eb5834df77a59e72a40442a228592dbe3196154 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6eb5834df7 ] NIFI-13067 Upgraded Spring Security from 6.2.3 to 6.2.4 - Upgraded Spring Boot from 3.2.4 to 3.2.5 for Registry - Upgraded OpenSAML from 4.3.0 to 4.3.1 - Upgraded AspectJ from 1.9.21 to 1.9.22 Signed-off-by: Pierre Villard This closes #8666. > Upgrade Spring Security to 6.2.4 > > > Key: NIFI-13067 > URL: https://issues.apache.org/jira/browse/NIFI-13067 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > Spring Security dependencies should be upgraded to > [6.2.4|https://github.com/spring-projects/spring-security/releases/tag/6.2.4] > to incorporate several minor bug fixes and transitive dependency upgrades. As > part of the upgrade, OpenSAML dependencies should be upgraded to 4.3.1 and > AspectJ should be upgraded to 1.9.22. > Spring Boot for Registry can also be incremented to > [3.2.5|https://github.com/spring-projects/spring-boot/releases/tag/v3.2.5] to > align with Spring Security 6.2.4 and Spring Framework 6.1.6. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13067) Upgrade Spring Security to 6.2.4
[ https://issues.apache.org/jira/browse/NIFI-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13067: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Spring Security to 6.2.4 > > > Key: NIFI-13067 > URL: https://issues.apache.org/jira/browse/NIFI-13067 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > Spring Security dependencies should be upgraded to > [6.2.4|https://github.com/spring-projects/spring-security/releases/tag/6.2.4] > to incorporate several minor bug fixes and transitive dependency upgrades. As > part of the upgrade, OpenSAML dependencies should be upgraded to 4.3.1 and > AspectJ should be upgraded to 1.9.22. > Spring Boot for Registry can also be incremented to > [3.2.5|https://github.com/spring-projects/spring-boot/releases/tag/v3.2.5] to > align with Spring Security 6.2.4 and Spring Framework 6.1.6. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13067 Upgrade Spring Security from 6.2.3 to 6.2.4 [nifi]
asfgit closed pull request #8666: NIFI-13067 Upgrade Spring Security from 6.2.3 to 6.2.4 URL: https://github.com/apache/nifi/pull/8666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13066) Upgrade Bouncy Castle to 1.78.1
[ https://issues.apache.org/jira/browse/NIFI-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13066: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Bouncy Castle to 1.78.1 > --- > > Key: NIFI-13066 > URL: https://issues.apache.org/jira/browse/NIFI-13066 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Bouncy Castle libraries should be upgraded to version > [1.78.1|https://www.bouncycastle.org/releasenotes.html#r1rv78] to incorporate > several bug fixes, which include resolving several vulnerabilities that apply > to usage patterns outside of NiFi integration points. > Bouncy Castle 1.78.1 corrects a dependency relationship issue with bcpg and > bcutil that caused issues with OpenPGP usage in version 1.78. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13066) Upgrade Bouncy Castle to 1.78.1
[ https://issues.apache.org/jira/browse/NIFI-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838966#comment-17838966 ] ASF subversion and git services commented on NIFI-13066: Commit 805ba9a8673ee669efb99470bd74620a2a0360c4 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=805ba9a867 ] NIFI-13066 Upgraded Bouncy Castle from 1.77 to 1.78.1 Signed-off-by: Pierre Villard This closes #8664. > Upgrade Bouncy Castle to 1.78.1 > --- > > Key: NIFI-13066 > URL: https://issues.apache.org/jira/browse/NIFI-13066 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Bouncy Castle libraries should be upgraded to version > [1.78.1|https://www.bouncycastle.org/releasenotes.html#r1rv78] to incorporate > several bug fixes, which include resolving several vulnerabilities that apply > to usage patterns outside of NiFi integration points. > Bouncy Castle 1.78.1 corrects a dependency relationship issue with bcpg and > bcutil that caused issues with OpenPGP usage in version 1.78. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13066) Upgrade Bouncy Castle to 1.78.1
[ https://issues.apache.org/jira/browse/NIFI-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838967#comment-17838967 ] ASF subversion and git services commented on NIFI-13066: Commit a8e34fd3f74f594bc840bd0c43d0b6ac109c2ffc in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a8e34fd3f7 ] NIFI-13066 Upgraded Bouncy Castle from 1.77 to 1.78.1 Signed-off-by: Pierre Villard This closes #8664. > Upgrade Bouncy Castle to 1.78.1 > --- > > Key: NIFI-13066 > URL: https://issues.apache.org/jira/browse/NIFI-13066 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Bouncy Castle libraries should be upgraded to version > [1.78.1|https://www.bouncycastle.org/releasenotes.html#r1rv78] to incorporate > several bug fixes, which include resolving several vulnerabilities that apply > to usage patterns outside of NiFi integration points. > Bouncy Castle 1.78.1 corrects a dependency relationship issue with bcpg and > bcutil that caused issues with OpenPGP usage in version 1.78. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13066 Upgrade Bouncy Castle from 1.77 to 1.78.1 [nifi]
asfgit closed pull request #8664: NIFI-13066 Upgrade Bouncy Castle from 1.77 to 1.78.1 URL: https://github.com/apache/nifi/pull/8664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org