[jira] [Commented] (NIFI-12845) ListenTCP/ListenTCPRecord: store the source IPAddress/Port
[ https://issues.apache.org/jira/browse/NIFI-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823434#comment-17823434 ] Beat Fuellemann commented on NIFI-12845: Hi Jim I checked that. It looks, that the IP address and port are the receiving part, that IP address and port we configure in the processor. So it does not help. > ListenTCP/ListenTCPRecord: store the source IPAddress/Port > -- > > Key: NIFI-12845 > URL: https://issues.apache.org/jira/browse/NIFI-12845 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Beat Fuellemann >Priority: Major > > We would like to use NIFI to handle a TCP Session, on which we recieve some > ASN1 keepalive. On the same session we should send back the Keepalive > response.^ > It would be great if we could store some additional attributes in the > flowfile. > * Source IP Address of the incoming connection > * Source Port of the incoming connection > This could be done for ListenTCP and ListenTCPRecord. > OR > It would be great if we have a HandleTCPRequest/HandleTCPResponse like > HandleHTTPRequest/HandleHTTPResponse. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12845) ListenTCP/ListenTCPRecord: store the source IPAddress/Port
[ https://issues.apache.org/jira/browse/NIFI-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-12845: Description: We would like to use NIFI to handle a TCP Session, on which we recieve some ASN1 keepalive. On the same session we should send back the Keepalive response.^ It would be great if we could store some additional attributes in the flowfile. * Source IP Address of the incoming connection * Source Port of the incoming connection This could be done for ListenTCP and ListenTCPRecord. OR It would be great if we have a HandleTCPRequest/HandleTCPResponse like HandleHTTPRequest/HandleHTTPResponse. was: We would like to use NIFI to handle a TCP Session, on which we recieve some ASN1 keepalive. On the same session we should send back the Keepalive response.^ It would be great if we could store some additional attributes in the flowfile. * Source IP Address of the incomming connection * Source Port of the incomming connection This could be done for ListenTCP and ListenTCPRecord. OR It would be great if we have a HandleTCPRequest/HandleTCPResponse like HandleHTTPRequest/HandleHTTPResponse. > ListenTCP/ListenTCPRecord: store the source IPAddress/Port > -- > > Key: NIFI-12845 > URL: https://issues.apache.org/jira/browse/NIFI-12845 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Beat Fuellemann >Priority: Major > > We would like to use NIFI to handle a TCP Session, on which we recieve some > ASN1 keepalive. On the same session we should send back the Keepalive > response.^ > It would be great if we could store some additional attributes in the > flowfile. > * Source IP Address of the incoming connection > * Source Port of the incoming connection > This could be done for ListenTCP and ListenTCPRecord. > OR > It would be great if we have a HandleTCPRequest/HandleTCPResponse like > HandleHTTPRequest/HandleHTTPResponse. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12864) Update NOTICE file to indicate the new class which now uses Spring's UriComponentsBuilder
[
https://issues.apache.org/jira/browse/NIFI-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12864:
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Update NOTICE file to indicate the new class which now uses Spring's
> UriComponentsBuilder
> -
>
> Key: NIFI-12864
> URL: https://issues.apache.org/jira/browse/NIFI-12864
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> With the changes made with NIFI-12832, the logic implemented from Spring's
> UriComponentsBuilder was moved to a different class. The NOTICE file should
> reflect the new class
> {code:java}
> nifi-api/src/main/java/org/apache/nifi/processor/util/URLValidator.java
> {code}
> instead of the old class
> {code:java}
> nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/UriUtils.java
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12864) Update NOTICE file to indicate the new class which now uses Spring's UriComponentsBuilder
[
https://issues.apache.org/jira/browse/NIFI-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823362#comment-17823362
]
ASF subversion and git services commented on NIFI-12864:
Commit 4083fd3f75252235fb9ff82ac1d0f12c3f903bdf in nifi's branch
refs/heads/main from dan-s1
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4083fd3f75 ]
NIFI-12864 Corrected NOTICE Reference to UriComponentsBuilder
This closes #8470
Signed-off-by: David Handermann
> Update NOTICE file to indicate the new class which now uses Spring's
> UriComponentsBuilder
> -
>
> Key: NIFI-12864
> URL: https://issues.apache.org/jira/browse/NIFI-12864
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> With the changes made with NIFI-12832, the logic implemented from Spring's
> UriComponentsBuilder was moved to a different class. The NOTICE file should
> reflect the new class
> {code:java}
> nifi-api/src/main/java/org/apache/nifi/processor/util/URLValidator.java
> {code}
> instead of the old class
> {code:java}
> nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/UriUtils.java
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
exceptionfactory closed pull request #8470: NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. URL: https://github.com/apache/nifi/pull/8470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823361#comment-17823361
]
ASF subversion and git services commented on NIFI-12645:
Commit 97516de1e0dbfa314a811c317094918cf65794d6 in nifi's branch
refs/heads/main from Jim Steinebrey
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=97516de1e0 ]
NIFI-12645 Fix to correctly invoke onStopped method of scripted processor
Signed-off-by: Matt Burgess
This closes #8471
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12645:
Fix Version/s: 2.0.0
1.26.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Fix For: 2.0.0, 1.26.0
>
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
exceptionfactory commented on PR #8470: URL: https://github.com/apache/nifi/pull/8470#issuecomment-1977493805 @dan-s1 The restart job option is available from the jobs view: https://github.com/apache/nifi/actions/runs/8145366866/job/22261448058?pr=8470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12645 Fix to correctly invoke onStopped method of scripted code [nifi]
mattyb149 closed pull request #8471: NIFI-12645 Fix to correctly invoke onStopped method of scripted code URL: https://github.com/apache/nifi/pull/8471 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12645 Fix to correctly invoke onStopped method of scripted code [nifi]
mattyb149 commented on PR #8471: URL: https://github.com/apache/nifi/pull/8471#issuecomment-1977491790 +1 LGTM, thanks for the fix! Merging to support/nifi-1.x and main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823356#comment-17823356
]
ASF subversion and git services commented on NIFI-12645:
Commit 7f2f5e77f65af1d688330092b4df9747f68ade50 in nifi's branch
refs/heads/support/nifi-1.x from Jim Steinebrey
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7f2f5e77f6 ]
NIFI-12645 Fix to correctly invoke onStopped method of scripted processor
Signed-off-by: Matt Burgess
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12740) Python Processors sometimes stuck in invalid state: 'Initializing runtime environment'
[
https://issues.apache.org/jira/browse/NIFI-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823358#comment-17823358
]
ASF subversion and git services commented on NIFI-12740:
Commit 55738f85221f9408f52878af28e21b2e79145868 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=55738f8522 ]
NIFI-12740 Fixed Python to Java Object Binding
Fixed issue in NiFiPythonGateway that stems from the fact that the thread
adding an object to the JavaObjectBindings was not necessarily the thread
removing them. The algorithm that was in place assumed that the same thread
would be used, in order to ensure that an object could be unbound before being
accessed. The new algorithm binds each new object to all active method
invocations and only unbinds the objects after all method invocations complete,
regardless of thread. Additionally, found that many method calls could create
new proxies on the Python side, just for getter methods whose values don't
change. This is very expensive, so introduced a new @Idempotent annotation that
can be added to interface methods such that we can cache the value and avoid
the expensive overhead.
This closes #8456
Signed-off-by: David Handermann
> Python Processors sometimes stuck in invalid state: 'Initializing runtime
> environment'
> --
>
> Key: NIFI-12740
> URL: https://issues.apache.org/jira/browse/NIFI-12740
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 2.0.0-M1, 2.0.0-M2
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When creating a Python processor, sometimes the Processor remains in an
> invalid state with the message "Initializing runtime environment"
> In the logs, we see the following error/stack trace:
> {code:java}
> 2024-02-05 17:23:30,308 ERROR [Initialize SetRecordField]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> VirtualThread[#123,Initialize
> SetRecordField]/runnable@ForkJoinPool-1-worker-5:
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> at
> org.apache.nifi.py4j.StandardPythonBridge.findExtensionId(StandardPythonBridge.java:322)
> at
> org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:99)
> at
> org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$3(StandardPythonBridge.java:142)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:73)
> at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12740) Python Processors sometimes stuck in invalid state: 'Initializing runtime environment'
[
https://issues.apache.org/jira/browse/NIFI-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-12740.
-
Resolution: Fixed
> Python Processors sometimes stuck in invalid state: 'Initializing runtime
> environment'
> --
>
> Key: NIFI-12740
> URL: https://issues.apache.org/jira/browse/NIFI-12740
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 2.0.0-M1, 2.0.0-M2
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When creating a Python processor, sometimes the Processor remains in an
> invalid state with the message "Initializing runtime environment"
> In the logs, we see the following error/stack trace:
> {code:java}
> 2024-02-05 17:23:30,308 ERROR [Initialize SetRecordField]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> VirtualThread[#123,Initialize
> SetRecordField]/runnable@ForkJoinPool-1-worker-5:
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> at
> org.apache.nifi.py4j.StandardPythonBridge.findExtensionId(StandardPythonBridge.java:322)
> at
> org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:99)
> at
> org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$3(StandardPythonBridge.java:142)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:73)
> at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12740: Fixed issue in NiFiPythonGateway that stems from the fact… [nifi]
exceptionfactory closed pull request #8456: NIFI-12740: Fixed issue in NiFiPythonGateway that stems from the fact… URL: https://github.com/apache/nifi/pull/8456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12645:
Affects Version/s: (was: 1.20.0)
(was: 1.19.1)
(was: 1.21.0)
(was: 1.22.0)
(was: 1.23.0)
(was: 1.24.0)
(was: 1.23.1)
(was: 1.23.2)
Status: Patch Available (was: Open)
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Steinebrey updated NIFI-12645:
--
Attachment: (was: Test_InvokeScriptedProcessor.json)
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.20.0, 1.19.1, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.23.1,
> 1.23.2
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Steinebrey updated NIFI-12645:
--
Attachment: Test_Script_Body_InvokeScriptedProcessor.json
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.20.0, 1.19.1, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.23.1,
> 1.23.2
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_Script_Body_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
dan-s1 commented on PR #8470: URL: https://github.com/apache/nifi/pull/8470#issuecomment-1977441472 @exceptionfactory I thought I had linked my Apache and Github accounts yet I do not see an option for restarting on my own PRs while I do see it for PRs of others. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
exceptionfactory commented on PR #8470: URL: https://github.com/apache/nifi/pull/8470#issuecomment-1977421144 > @exceptionfactory Can ci-workflow / Ubuntu Corretto JDK 21 EN (pull_request) be restarted? The failure has nothing to do with my change as my change was only on the NOTICE file and not on any code. Thanks! @dan-s1 You should be able to restart GitHub Actions after linking your Apache account to your GitHub account, I can provide additional details if needed. In this case, however, it is less necessary, since this is a documentation change, and the other workflows completed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
dan-s1 commented on PR #8470: URL: https://github.com/apache/nifi/pull/8470#issuecomment-1977358193 @exceptionfactory Can ci-workflow / Ubuntu Corretto JDK 21 EN (pull_request) be restarted? The failure has nothing to do with my change as my change was only on the NOTICE file and not on any code. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12865 BUG - Checkboxes are inconsistently styled with primary or accent [nifi]
james-elliott opened a new pull request, #8472: URL: https://github.com/apache/nifi/pull/8472 Updated all the checkboxes to use color="primary" to keep them consistent. As you can see in the below image, the checkboxes don't match. This PR changes all checkboxes to use the primary theme color.  # Summary [NIFI-12865](https://issues.apache.org/jira/browse/NIFI-12865) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12865) BUG - Checkboxes are inconsistently styled with primary or accent
[ https://issues.apache.org/jira/browse/NIFI-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Elliott reassigned NIFI-12865: Assignee: James Elliott > BUG - Checkboxes are inconsistently styled with primary or accent > - > > Key: NIFI-12865 > URL: https://issues.apache.org/jira/browse/NIFI-12865 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: James Elliott >Assignee: James Elliott >Priority: Minor > Attachments: image.png > > > As you can see in the attached image, the checkboxes for the relationship > options are styled with both primary and accent colors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12865) BUG - Checkboxes are inconsistently styled with primary or accent
James Elliott created NIFI-12865: Summary: BUG - Checkboxes are inconsistently styled with primary or accent Key: NIFI-12865 URL: https://issues.apache.org/jira/browse/NIFI-12865 Project: Apache NiFi Issue Type: Sub-task Reporter: James Elliott Attachments: image.png As you can see in the attached image, the checkboxes for the relationship options are styled with both primary and accent colors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12645 Fix to correctly invoke onStopped method of scripted code [nifi]
jrsteinebrey opened a new pull request, #8471: URL: https://github.com/apache/nifi/pull/8471 NIFI-12645 Fix to correctly invoke onStopped method of scripted processor # Summary [NIFI-12645](https://issues.apache.org/jira/browse/NIFI-12645) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-12645) issue created ### Pull Request Tracking - [ X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-12645` - [ X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-12645` ### Pull Request Formatting - [ X] Pull Request based on current revision of the `main` branch - [ X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. I tested the new code from the NIFI user interface by creating a flow which used InvokeScriptedProcessor. I the scripted processor, I made its onStopped method write a statement to the log. Then a started and stopped the processor andI saw that the log statement wasa succesfully written to the log file. ### Build - [ X] Build completed using `mvn clean install -P contrib-check` - [X ] JDK 21 ### Licensing No new files added and no new dependencies added. ### Documentation No documentation changes needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823292#comment-17823292
]
Jim Steinebrey commented on NIFI-12645:
---
Test_InvokeScriptedProcessor.json creates a process group for testing this
issue. After the fix, stopping the InvokeScriptedProcessor will cause the
scripted processor to write to the log "NO ERROR: groovy onStopped was called"
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.20.0, 1.19.1, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.23.1,
> 1.23.2
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Steinebrey updated NIFI-12645:
--
Attachment: Test_InvokeScriptedProcessor.json
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.20.0, 1.19.1, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.23.1,
> 1.23.2
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: Test_InvokeScriptedProcessor.json,
> image-2024-01-19-12-40-24-453.png, image-2024-01-19-12-41-15-173.png,
> image-2024-01-19-12-44-08-030.png, image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12645) InvokeScriptedProcessor: The onStopped method is never called
[
https://issues.apache.org/jira/browse/NIFI-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Steinebrey reassigned NIFI-12645:
-
Assignee: Jim Steinebrey
> InvokeScriptedProcessor: The onStopped method is never called
> -
>
> Key: NIFI-12645
> URL: https://issues.apache.org/jira/browse/NIFI-12645
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.20.0, 1.19.1, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.23.1,
> 1.23.2
> Environment: All operatiing system
>Reporter: Antonio Pezzuto
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: easyfix
> Attachments: image-2024-01-19-12-40-24-453.png,
> image-2024-01-19-12-41-15-173.png, image-2024-01-19-12-44-08-030.png,
> image-2024-01-19-12-44-39-464.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Processor: *InvokeScriptedProcessor*
> Script Engine : groovy
> The _InvokeScriptedProcessor_ processor was used to create a custom processor
> in groovy.
> The groovy custom processor exposes the *onStopped* method
>
> {code:java}
> public void onStopped(ProcessContext context) throws Exception {
> System.out.println("Stop")
> restServer?.shutDown()
> }{code}
>
> When the _InvokeScriptedProcessor_ is stopped the groovy processor's
> *onStopped* method is not invoked.
> I ran the InvokeScriptedProcessor processor in remote debug and the cause
> seems to be due to a bug in the management of the scriptNeedsReload variable.
> The groovy processor's onStopped method executes only if the
> scriptNeedsReload instance variable equals false
> !image-2024-01-19-12-41-15-173.png!
> According to my analysis _scriptNeedsReload_ is always set to true due to an
> error in the *reloadScript* method.
> Currently the *reloadScript* method returns true if the script body is empty
> or if there are validation errors, it returns false if there are no
> validation errors.
> The *reloadScript* method should return true if there are no validation
> errors and return false if the script body is empty or if there are
> validation errors.
> !image-2024-01-19-12-44-08-030.png!
> !image-2024-01-19-12-44-39-464.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12864) Update NOTICE file to indicate the new class which now uses Spring's UriComponentsBuilder
[
https://issues.apache.org/jira/browse/NIFI-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Stieglitz updated NIFI-12864:
Status: Patch Available (was: Open)
> Update NOTICE file to indicate the new class which now uses Spring's
> UriComponentsBuilder
> -
>
> Key: NIFI-12864
> URL: https://issues.apache.org/jira/browse/NIFI-12864
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the changes made with NIFI-12832, the logic implemented from Spring's
> UriComponentsBuilder was moved to a different class. The NOTICE file should
> reflect the new class
> {code:java}
> nifi-api/src/main/java/org/apache/nifi/processor/util/URLValidator.java
> {code}
> instead of the old class
> {code:java}
> nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/UriUtils.java
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12864 Corrected the NOTICE file to refer to the correct file which uses Spring's UriComponentsBuilder. [nifi]
dan-s1 opened a new pull request, #8470: URL: https://github.com/apache/nifi/pull/8470 # Summary [NIFI-12864](https://issues.apache.org/jira/browse/NIFI-12864) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11107 In ConsumeIMAP and ConsumePOP3 added support for OAUTH based authorization. [nifi]
RomanOttGmx commented on PR #6900:
URL: https://github.com/apache/nifi/pull/6900#issuecomment-1977185841
In IMAP Processor buildUrl will be triggered on creation of new receiver.
```
`protected ImapMailReceiver buildMessageReceiver(ProcessContext
processContext) {
ImapMailReceiver receiver = new
ImapMailReceiver(this.buildUrl(processContext));`
```
And the buildMessageReceiver is requested here in AbstractEmailProcessor
```
private synchronized void initializeIfNecessary(ProcessContext context,
ProcessSession processSession) {
if (this.messageReceiver == null) {
this.processSession = processSession;
this.messageReceiver = this.buildMessageReceiver(context);
```
This is called in onTrigger method of AbstractEmailProcessor, but the
receiver still exists since start up, so buildUrl will never be requested again
after enabling the ConsumeImapProcessor.
```
public void onTrigger(ProcessContext context, ProcessSession processSession)
throws ProcessException {
this.initializeIfNecessary(context, processSession);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-11107 In ConsumeIMAP and ConsumePOP3 added support for OAUTH based authorization. [nifi]
RomanOttGmx commented on PR #6900: URL: https://github.com/apache/nifi/pull/6900#issuecomment-1977172784 Yes, implement ConsumeImapProcessor with OAUTH2 and an associated oauth2AccessTokenProviderService. This Provider is only triggered on the onScheduled Method of the AbstractEmailProcessor and oauth2AccessTokenProvider.getAccessDetails() to refresh the token is only requested also on onScheduledMethod. But in onTriggered Method this will not trigger the oauth2AccessTokenProvider.getAccessDetails() We startet a local debug with break points. Configured the ConsumeImapProcessor with Schedule "Cron-Based" every 5 minutes. onScheduled => triggered on enabling the Processor Every 5 minutes there is the onTriggered Method triggered and not the onScheduled. That is general for all Processors which use the AbstractEmailProcessor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12864) Update NOTICE file to indicate the new class which now uses Spring's UriComponentsBuilder
Daniel Stieglitz created NIFI-12864:
---
Summary: Update NOTICE file to indicate the new class which now
uses Spring's UriComponentsBuilder
Key: NIFI-12864
URL: https://issues.apache.org/jira/browse/NIFI-12864
Project: Apache NiFi
Issue Type: Improvement
Reporter: Daniel Stieglitz
Assignee: Daniel Stieglitz
With the changes made with NIFI-12832, the logic implemented from Spring's
UriComponentsBuilder was moved to a different class. The NOTICE file should
reflect the new class
{code:java}
nifi-api/src/main/java/org/apache/nifi/processor/util/URLValidator.java
{code}
instead of the old class
{code:java}
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/UriUtils.java
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (MINIFICPP-2310) Extend ListenTCP by new ways to delimit flow files
Marton Szasz created MINIFICPP-2310: --- Summary: Extend ListenTCP by new ways to delimit flow files Key: MINIFICPP-2310 URL: https://issues.apache.org/jira/browse/MINIFICPP-2310 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Marton Szasz * custom message delimiter * option to separate by connection (no delimiter, only optional size limit) * batching multiple messages to 1 flow file (like NiFi 'Max Batch Size') -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12863) Extend ListenTCP by new ways to delimit flow files
[
https://issues.apache.org/jira/browse/NIFI-12863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved NIFI-12863.
-
Resolution: Invalid
I meant to create this under MINIFICPP
> Extend ListenTCP by new ways to delimit flow files
> --
>
> Key: NIFI-12863
> URL: https://issues.apache.org/jira/browse/NIFI-12863
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Marton Szasz
>Priority: Major
>
> - custom message delimiter
> - option to separate by connection (no delimiter, only optional size limit)
> - batching multiple messages to 1 flow file (like NiFi '{*}Max Batch Size{*}')
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (NIFI-12863) Extend ListenTCP by new ways to delimit flow files
[
https://issues.apache.org/jira/browse/NIFI-12863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed NIFI-12863.
---
Assignee: Marton Szasz
> Extend ListenTCP by new ways to delimit flow files
> --
>
> Key: NIFI-12863
> URL: https://issues.apache.org/jira/browse/NIFI-12863
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
>
> - custom message delimiter
> - option to separate by connection (no delimiter, only optional size limit)
> - batching multiple messages to 1 flow file (like NiFi '{*}Max Batch Size{*}')
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12863) Extend ListenTCP by new ways to delimit flow files
Marton Szasz created NIFI-12863:
---
Summary: Extend ListenTCP by new ways to delimit flow files
Key: NIFI-12863
URL: https://issues.apache.org/jira/browse/NIFI-12863
Project: Apache NiFi
Issue Type: New Feature
Reporter: Marton Szasz
- custom message delimiter
- option to separate by connection (no delimiter, only optional size limit)
- batching multiple messages to 1 flow file (like NiFi '{*}Max Batch Size{*}')
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12861 Provide documentation on JASN1Reader that it only supports… [nifi]
Kapkiai opened a new pull request, #8469: URL: https://github.com/apache/nifi/pull/8469 … JDK and not just JRE # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12704) Record Path function escapeJson raises NPE when given root "/" as argument
[
https://issues.apache.org/jira/browse/NIFI-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Stieglitz updated NIFI-12704:
Description:
My use case is I want to create a field called "original" which is the escaped
string of my whole record. This will preserve the original contents of the
message before I start transforming it.
My expectation is that I should be able to use an UpdateRecord processor to
create the field using RecordPath language.
What actually happens is that when I use
{code:java}
escapeJson(/)
{code}
as the function the result is that the processor throws a Null Pointer
Exception (NPE).
Detail:
For any input flow file containing JSON records, define an UpdateRecord
processor with these settings.
* Replacement Value Strategy = Record Path Value
* /original = escapeJson(/)
When the flow file is presented to the processor, the processor generates an
NPE.
For example, I present this record.
{noformat}
[{"hello":"world","record":{"key":"one","value":"hello","subrecord":{"key":"two","value":"bob"}},"array":[0,1,2]}]{noformat}
If the escapeJson function is offered an existing field to escape then it works
as expected. Stack trace as follows.
{noformat}
2024-01-31 12:17:15 2024-01-31 12:17:14,741 ERROR [Timer-Driven Process
Thread-8] o.a.n.processors.standard.UpdateRecord
UpdateRecord[id=5f6b498d-018d-1000--323c6b89] Failed to process
StandardFlowFileRecord[uuid=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1706703050563-1, container=default,
section=1], offset=114,
length=114],offset=0,name=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,size=114]; will
route to failure
2024-01-31 12:17:15 org.apache.nifi.record.path.exception.RecordPathException:
java.lang.NullPointerException
2024-01-31 12:17:15 at
org.apache.nifi.record.path.RecordPath.compile(RecordPath.java:105)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.ConcurrentHashMap.compute(Unknown Source)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108)
2024-01-31 12:17:15 at
com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:56)
2024-01-31 12:17:15 at
org.apache.nifi.record.path.util.RecordPathCache.getCompiled(RecordPathCache.java:34)
2024-01-31 12:17:15 at
org.apache.nifi.processors.standard.UpdateRecord.process(UpdateRecord.java:166)
2024-01-31 12:17:15 at
org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:147)
2024-01-31 12:17:15 at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3432)
2024-01-31 12:17:15 at
org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
2024-01-31 12:17:15 at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
2024-01-31 12:17:15 at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
2024-01-31 12:17:15 at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
2024-01-31 12:17:15 at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
2024-01-31 12:17:15 at
org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
2024-01-31 12:17:15 at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
2024-01-31 12:17:15 at java.base/java.lang.Thread.run(Unknown Source)
2024-01-31 12:17:15 Caused by: java.lang.NullPointerException: null
2024-01-31 12:17:15 at
org.apache.nifi.record.path.paths.RecordPathCompiler.buildPath(RecordPathCompiler.java:131)
2024-01-31 12:17:15 at
org.apache.nifi.record.path.paths.RecordPathCompiler.getArgPaths(RecordPathCompiler.java:511)
2024-01-31 12:17:15 at
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511473149
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -129,31 +172,35 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
}
}
-private static StreamCallback getStreamCallback(final boolean encode,
final String encoding) {
-if (encode) {
-if (encoding.equalsIgnoreCase(BASE64_ENCODING)) {
-return new EncodeBase64();
-} else if (encoding.equalsIgnoreCase(BASE32_ENCODING)) {
-return new EncodeBase32();
-} else {
-return new EncodeHex();
-}
-} else {
-if (encoding.equalsIgnoreCase(BASE64_ENCODING)) {
-return new DecodeBase64();
-} else if (encoding.equalsIgnoreCase(BASE32_ENCODING)) {
-return new DecodeBase32();
-} else {
-return new DecodeHex();
-}
+private static StreamCallback getStreamCallback(final boolean encode,
final EncodingType encoding,
+final int lineLength, final String lineSeparator) {
+switch(encoding) {
+case BASE64:
+return encode ? new EncodeBase64(lineLength, lineSeparator) :
new DecodeBase64();
+case BASE32:
+return encode ? new EncodeBase32(lineLength, lineSeparator) :
new DecodeBase32();
+default:
+return encode ? new EncodeHex() : new DecodeHex();
}
}
private static class EncodeBase64 implements StreamCallback {
+private int lineLength;
+private String lineSeparator;
Review Comment:
Intellij points out that these should be final
```suggestion
private final int lineLength;
private final String lineSeparator;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] NIFI-12860 Null exception from ExtensionMetadata caused by NIFI-12113 [nifi]
pradden opened a new pull request, #8468: URL: https://github.com/apache/nifi/pull/8468 # Summary [NIFI-12860](https://issues.apache.org/jira/browse/NIFI-12860) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511473945
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -171,9 +218,22 @@ public void process(final InputStream in, final
OutputStream out) throws IOExcep
private static class EncodeBase32 implements StreamCallback {
+private int lineLength;
+private String lineSeparator;
Review Comment:
```suggestion
private final int lineLength;
private final String lineSeparator;
```
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -129,31 +172,35 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
}
}
-private static StreamCallback getStreamCallback(final boolean encode,
final String encoding) {
-if (encode) {
-if (encoding.equalsIgnoreCase(BASE64_ENCODING)) {
-return new EncodeBase64();
-} else if (encoding.equalsIgnoreCase(BASE32_ENCODING)) {
-return new EncodeBase32();
-} else {
-return new EncodeHex();
-}
-} else {
-if (encoding.equalsIgnoreCase(BASE64_ENCODING)) {
-return new DecodeBase64();
-} else if (encoding.equalsIgnoreCase(BASE32_ENCODING)) {
-return new DecodeBase32();
-} else {
-return new DecodeHex();
-}
+private static StreamCallback getStreamCallback(final boolean encode,
final EncodingType encoding,
+final int lineLength, final String lineSeparator) {
+switch(encoding) {
+case BASE64:
+return encode ? new EncodeBase64(lineLength, lineSeparator) :
new DecodeBase64();
+case BASE32:
+return encode ? new EncodeBase32(lineLength, lineSeparator) :
new DecodeBase32();
+default:
+return encode ? new EncodeHex() : new DecodeHex();
}
}
private static class EncodeBase64 implements StreamCallback {
+private int lineLength;
+private String lineSeparator;
Review Comment:
Intellij points out these should be final
```suggestion
private final int lineLength;
private final String lineSeparator;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1977041853 In the same Windows environment I had issues with `vswhere`, I also had issue with the compiler command line after installing vswhere with scoop and refusing the offered choco install. ``` -- The C compiler identification is unknown -- The CXX compiler identification is unknown CMake Error at CMakeLists.txt:25 (project): The CMAKE_C_COMPILER: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe is not a full path to an existing compiler tool. Tell CMake where to find the compiler by setting either the environment variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH. CMake Error at CMakeLists.txt:25 (project): The CMAKE_CXX_COMPILER: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe is not a full path to an existing compiler tool. Tell CMake where to find the compiler by setting either the environment variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH. -- Configuring incomplete, errors occurred! ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511474443
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
[jira] [Updated] (NIFI-12860) Null exception from ExtensionMetadata caused by NIFI-12113
[ https://issues.apache.org/jira/browse/NIFI-12860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Radden updated NIFI-12860: Description: Shown using command similar to: cli.sh registry list-extensions -ot json -p tk.properties ERROR: Error executing command 'list-extensions' : null was: Shown using command similar to: cli.sh registry list-extensions -ot json -p tk.properties ERROR: Error executing command 'list-extensions' : null https://github.com/apache/nifi/pull/8467 > Null exception from ExtensionMetadata caused by NIFI-12113 > -- > > Key: NIFI-12860 > URL: https://issues.apache.org/jira/browse/NIFI-12860 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Peter Radden >Priority: Minor > > Shown using command similar to: > cli.sh registry list-extensions -ot json -p tk.properties > ERROR: Error executing command 'list-extensions' : null > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12862) FlowAnalysisResults should not leak anauthorized component details
[ https://issues.apache.org/jira/browse/NIFI-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Palfy updated NIFI-12862: --- Summary: FlowAnalysisResults should not leak anauthorized component details (was: FlowAnalysisResults should leak anauthorized component details) > FlowAnalysisResults should not leak anauthorized component details > -- > > Key: NIFI-12862 > URL: https://issues.apache.org/jira/browse/NIFI-12862 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > > The FlowAnalysisResultEntity hold FlowAnalysisRuleViolationDTO that contain > the name of a violating component as a message describing the violation. This > usually contains details about the violating component. > A user can see these even if they don't have read permission for that > particular component. > In clustered environment the request merger filters out such violations but > in a non-clustered environment there is no such filtering phase. > The FlowAnalysisRuleViolationDTO itself should be built accordingly and leave > certain details blank when the user lacks read permissions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11520: Add a menu to display Flow Analysis report results [nifi]
tpalfy commented on code in PR #8273:
URL: https://github.com/apache/nifi/pull/8273#discussion_r1511380014
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/controllers/nf-ng-canvas-flow-status-controller.js:
##
@@ -400,6 +406,556 @@
}
}
+/**
+ * The flow analysis controller.
+ */
+
+this.flowAnalysis = {
+
+/**
+ * Create the list of rule violations
+ */
+buildRuleViolationsList: function(rules, violationsAndRecs) {
+var ruleViolationCountEl = $('#rule-violation-count');
+var ruleViolationListEl = $('#rule-violations-list');
+var ruleWarningCountEl = $('#rule-warning-count');
+var ruleWarningListEl = $('#rule-warnings-list');
+var violations = violationsAndRecs.filter(function
(violation) {
+return violation.enforcementPolicy === 'ENFORCE'
+});
+var warnings = violationsAndRecs.filter(function
(violation) {
+return violation.enforcementPolicy === 'WARN'
+});
+ruleViolationCountEl.empty().text('(' + violations.length
+ ')');
+ruleWarningCountEl.empty().text('(' + warnings.length +
')');
+ruleViolationListEl.empty();
+violations.forEach(function(violation) {
+var rule = rules.find(function(rule) {
+return rule.id === violation.ruleId;
+});
+// create DOM elements
+var violationListItemEl = $('');
+var violationEl = $('');
+var violationListItemWrapperEl = $('');
+var violationRuleEl = $('');
+var violationListItemNameEl = $('');
+var violationListItemIdEl = $('');
+var violationInfoButtonEl = $('');
+
+// add text content and button data
+$(violationRuleEl).text(rule.name);
+
$(violationListItemNameEl).text(violation.subjectDisplayName);
Review Comment:
@mcgilman I have opened https://issues.apache.org/jira/browse/NIFI-12862 and
going to open a PR for the backend to fix this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-12862) FlowAnalysisResults should leak anauthorized component details
[ https://issues.apache.org/jira/browse/NIFI-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Palfy reassigned NIFI-12862: -- Assignee: Tamas Palfy > FlowAnalysisResults should leak anauthorized component details > -- > > Key: NIFI-12862 > URL: https://issues.apache.org/jira/browse/NIFI-12862 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > > The FlowAnalysisResultEntity hold FlowAnalysisRuleViolationDTO that contain > the name of a violating component as a message describing the violation. This > usually contains details about the violating component. > A user can see these even if they don't have read permission for that > particular component. > In clustered environment the request merger filters out such violations but > in a non-clustered environment there is no such filtering phase. > The FlowAnalysisRuleViolationDTO itself should be built accordingly and leave > certain details blank when the user lacks read permissions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12862) FlowAnalysisResults should leak anauthorized component details
Tamas Palfy created NIFI-12862: -- Summary: FlowAnalysisResults should leak anauthorized component details Key: NIFI-12862 URL: https://issues.apache.org/jira/browse/NIFI-12862 Project: Apache NiFi Issue Type: Bug Reporter: Tamas Palfy The FlowAnalysisResultEntity hold FlowAnalysisRuleViolationDTO that contain the name of a violating component as a message describing the violation. This usually contains details about the violating component. A user can see these even if they don't have read permission for that particular component. In clustered environment the request merger filters out such violations but in a non-clustered environment there is no such filtering phase. The FlowAnalysisRuleViolationDTO itself should be built accordingly and leave certain details blank when the user lacks read permissions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12720 BUG - Dark mode skeleton loader styles [nifi]
james-elliott commented on PR #8465: URL: https://github.com/apache/nifi/pull/8465#issuecomment-1976857294 > @james-elliott Can you please add a more descriptive title to this PR? Updated the title to match the JIRA issue. I also added before and after screenshots -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12720 [nifi]
dan-s1 commented on PR #8465: URL: https://github.com/apache/nifi/pull/8465#issuecomment-1976838181 @james-elliott Can you please add a more descriptive title to this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511339486
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
Review Comment:
@knguyen1 Thanks for pointing out those classes.
@exceptionfactory Based on the above classes there is another property
which I am not sure should be
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511339486
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
Review Comment:
@knguyen1 Thanks for pointing out those classes.
@exceptionfactory Based on the above classes there is another property
which I am not sure should be
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511285858
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -237,4 +297,14 @@ public void process(final InputStream in, final
OutputStream out) throws IOExcep
out.flush();
}
}
+
+private static EncodingType getEncodingTypeFromString(final String
encodingTypeValue) {
+if (encodingTypeValue.compareToIgnoreCase("base64") == 0) {
+return EncodingType.BASE64;
+} else if (encodingTypeValue.compareToIgnoreCase("base32") == 0) {
+return EncodingType.BASE32;
Review Comment:
No need to hard code the values again when they are already defined in the
`EncodingType` enum. Also, ignoring the case I do not believe is needed at
least from UI since the values to choose from are only the ones in the
`EncodingType` enum. If it is still needed (perhaps if the processor is
manipulated through the REST API) then simply use `equalsIgnoreCase`.
```suggestion
if (EncodingType.BASE64.getValue().equals(encodingTypeValue)) {
return EncodingType.BASE64;
} else if (EncodingType.BASE32.getValue().equals(encodingTypeValue))
{
return EncodingType.BASE32;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511285858
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -237,4 +297,14 @@ public void process(final InputStream in, final
OutputStream out) throws IOExcep
out.flush();
}
}
+
+private static EncodingType getEncodingTypeFromString(final String
encodingTypeValue) {
+if (encodingTypeValue.compareToIgnoreCase("base64") == 0) {
+return EncodingType.BASE64;
+} else if (encodingTypeValue.compareToIgnoreCase("base32") == 0) {
+return EncodingType.BASE32;
Review Comment:
No need to hard code the values again when they are already defined in the
`EncodingType` enum.
```suggestion
if (EncodingType.BASE64.getValue().equals(encodingTypeValue)) {
return EncodingType.BASE64;
} else if (EncodingType.BASE32.getValue().equals(encodingTypeValue))
{
return EncodingType.BASE32;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511285858
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -237,4 +297,14 @@ public void process(final InputStream in, final
OutputStream out) throws IOExcep
out.flush();
}
}
+
+private static EncodingType getEncodingTypeFromString(final String
encodingTypeValue) {
+if (encodingTypeValue.compareToIgnoreCase("base64") == 0) {
+return EncodingType.BASE64;
+} else if (encodingTypeValue.compareToIgnoreCase("base32") == 0) {
+return EncodingType.BASE32;
Review Comment:
```suggestion
if (EncodingType.BASE64.getValue().equals(encodingTypeValue)) {
return EncodingType.BASE64;
} else if (EncodingType.BASE32.getValue().equals(encodingTypeValue))
{
return EncodingType.BASE32;
```
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -237,4 +297,14 @@ public void process(final InputStream in, final
OutputStream out) throws IOExcep
out.flush();
}
}
+
+private static EncodingType getEncodingTypeFromString(final String
encodingTypeValue) {
Review Comment:
No need to add suffix `FromString` as it is obvious from the argument.
```suggestion
private static EncodingType getEncodingType(final String
encodingTypeValue) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
knguyen1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511280955
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
Review Comment:
@dan-s1 `Base32OutputStream` and `Base64OutputStream`. Look for the
argument `lineSeparator`. The thinking at the time was: if it's available in
the
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511277613
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
dan-s1 commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511275461
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
Review Comment:
@exceptionfactory Thank you for including me. I do not have an opinion
really either way on this.
@knguyen1 Can you clarify what you meant by
>
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511242466 ## .github/workflows/ci.yml: ## @@ -75,12 +131,72 @@ jobs: timeout-minutes: 240 env: LUA_DIR: D:\a\nifi-minifi-cpp\nifi-minifi-cpp\.lua + WINDOWS_MINIFI_OPTIONS: >- +-DCMAKE_BUILD_TYPE=Release +-DBUILD_ROCKSDB=ON +-DBUILD_SHARED_LIBS=OFF +-DCI_BUILD=ON +-DCUSTOM_MALLOC=OFF +-DDOCKER_BUILD_ONLY=OFF +-DDOCKER_PUSH=OFF +-DDOCKER_SKIP_TESTS=ON +-DENABLE_ALL=OFF +-DENABLE_AWS=ON +-DENABLE_AZURE=ON +-DENABLE_BUSTACHE=OFF +-DENABLE_BZIP2=ON +-DENABLE_CIVET=ON +-DENABLE_COAP=OFF +-DENABLE_CONTROLLER=OFF Review Comment: Now that you fixed its build, I think we should enable the controller even on Windows. ## bootstrap/py_bootstrap.bat: ## @@ -0,0 +1,29 @@ +@echo off + +REM Check if Python is installed +where python > nul 2>&1 +if %errorlevel% neq 0 ( +echo Python is not installed +exit /b 1 +) + +REM Check if venv module is available +python -m venv --help > nul 2>&1 +if %errorlevel% neq 0 ( +echo venv module is not available +exit /b 1 +) + +set "SCRIPT_DIR=%~dp0" +set "VENV_DIR=%SCRIPT_DIR%venv" + +if exist "%VENV_DIR%" ( +call "%VENV_DIR%\Scripts\activate" +) else ( +echo Creating virtualenv +python -m venv "%VENV_DIR%" +call "%VENV_DIR%\Scripts\activate" Review Comment: Aren't we supposed to call activate.bat here? Why does this even work? ## Windows.md: ## @@ -120,7 +120,7 @@ A basic working CMake configuration can be inferred from the `win_build_vs.bat`. ``` mkdir build cd build -cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DUSE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DOPENSSL_OFF=OFF -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON -DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DENABLE_WEL=ON -DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. +cmake -G "Visual Studio 16 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DMINIFI_USE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DMINIFI_OPENSSL=ON -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DUSE_SYSTEM_UUID=OFF -DENABLE_LIBARCHIVE=ON -DENABLE_WEL=ON -DMINIFI_FAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. Review Comment: On my machine, there's `Visual Studio 16 2019` and `Visual Studio 17 2022`, but no `Visual Studio 16`. Was this change made by accident? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12861) Provide documentation on JASN1Reader that it only supports JDK and not just JRE
Mathew Kapkiai created NIFI-12861: - Summary: Provide documentation on JASN1Reader that it only supports JDK and not just JRE Key: NIFI-12861 URL: https://issues.apache.org/jira/browse/NIFI-12861 Project: Apache NiFi Issue Type: Improvement Reporter: Mathew Kapkiai Assignee: Mathew Kapkiai -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511239034 ## extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp: ## Review Comment: thx for fixing this, I also ran into this issue a few weeks ago. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2283 Create tool to encrypt sensitive properties in config.yml [nifi-minifi-cpp]
szaszm commented on code in PR #1725:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1725#discussion_r1511219340
##
libminifi/src/core/json/JsonFlowSerializer.cpp:
##
@@ -47,14 +47,21 @@ void
JsonFlowSerializer::encryptSensitiveProperties(rapidjson::Value _j
}
if (properties.at(name).isSensitive()) {
auto& value = property.value;
- const std::string_view value_sv{value.GetString(),
value.GetStringLength()};
+ const std::string_view value_sv = component_overrides.contains(name) ?
component_overrides.at(name) : std::string_view{value.GetString(),
value.GetStringLength()};
const std::string encrypted_value =
utils::crypto::property_encryption::encrypt(value_sv, encryption_provider);
value.SetString(encrypted_value.c_str(), encrypted_value.size(), alloc);
}
+component_overrides.erase(name);
+ }
+
+ for (const auto& [name, value] : component_overrides) {
+const std::string encrypted_value =
utils::crypto::property_encryption::encrypt(value, encryption_provider);
+property_jsons.AddMember(rapidjson::Value(name, alloc),
rapidjson::Value(encrypted_value, alloc), alloc);
Review Comment:
What's the idea behind overrides? I see that the override values are used in
plaintext to override existing properties in the flow definition, but if they
were not used, then they are used encrypted. I don't understand why it works
that way.
##
libminifi/include/core/ProcessGroup.h:
##
@@ -205,6 +205,8 @@ class ProcessGroup : public CoreComponent {
*/
std::shared_ptr
findControllerService(const std::string ) const;
+ std::vector>
getAllControllerServices() const;
Review Comment:
Does this really need to give ownership to the callers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Null exception from ExtensionMetadata caused by NIFI-12113 [nifi]
dan-s1 commented on PR #8467: URL: https://github.com/apache/nifi/pull/8467#issuecomment-1976660463 @pradden Thank you for first time contribution! We hope the first of many. As part of the committing process there is a check list of items. One of those items is > Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-0 Can you please edit the title of this PR to have the title start with the NIFI-12860? Also under the summary there should be link to NIFI-12860. Thank you for your help in this matter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2282 Support re-encryption of sensitive properties [nifi-minifi-cpp]
szaszm commented on code in PR #1739:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1739#discussion_r1511197680
##
encrypt-config/FlowConfigEncryptor.cpp:
##
@@ -61,114 +62,150 @@ std::vector
listSensitiveProperties(const minifi::core::Proce
std::vector processors;
process_group.getAllProcessors(processors);
- for (const auto *processor : processors) {
+ for (const auto* processor : processors) {
gsl_Expects(processor);
for (const auto& [_, property] : processor->getProperties()) {
if (property.isSensitive()) {
sensitive_properties.push_back(SensitiveProperty{
-.type = Type::Processor,
+.component_type = ComponentType::Processor,
.component_id = processor->getUUID(),
.component_name = processor->getName(),
.property_name = property.getName(),
-.property_display_name = property.getDisplayName()});
+.property_display_name = property.getDisplayName(),
+.property_value = property.getValue().to_string()});
}
}
}
- for (const auto _service_node :
process_group.getAllControllerServices()) {
+ for (const auto& controller_service_node :
process_group.getAllControllerServices()) {
gsl_Expects(controller_service_node);
const auto controller_service =
controller_service_node->getControllerServiceImplementation();
gsl_Expects(controller_service);
for (const auto& [_, property] : controller_service->getProperties()) {
if (property.isSensitive()) {
sensitive_properties.push_back(SensitiveProperty{
-.type = Type::ControllerService,
+.component_type = ComponentType::ControllerService,
.component_id = controller_service->getUUID(),
.component_name = controller_service->getName(),
.property_name = property.getName(),
-.property_display_name = property.getDisplayName()});
+.property_display_name = property.getDisplayName(),
+.property_value = property.getValue().to_string()});
}
}
}
return sensitive_properties;
}
-template
-void encryptSensitiveValuesInFlowConfigImpl(
-const minifi::encrypt_config::EncryptionKeys& keys, const
std::filesystem::path& minifi_home, const std::filesystem::path&
flow_config_path, Func create_overrides) {
- const auto configure = std::make_shared();
+std::unordered_map> createOverridesInteractively(const
std::vector& sensitive_properties) {
+ std::unordered_map> overrides;
+ std::cout << '\n';
+ for (const auto& sensitive_property : sensitive_properties) {
+std::cout << magic_enum::enum_name(sensitive_property.component_type) << "
" << sensitive_property.component_name << " (" <<
sensitive_property.component_id.to_string() << ") "
+ << "has sensitive property " <<
sensitive_property.property_display_name << "\nenter a new value or press
Enter to keep the current value unchanged: ";
+std::cout.flush();
+std::string new_value;
+std::getline(std::cin, new_value);
+if (!new_value.empty()) {
+
overrides[sensitive_property.component_id].emplace(sensitive_property.property_name,
new_value);
+}
+ }
+ return overrides;
+}
+
+std::unordered_map> createOverridesForSingleProperty(
+const std::vector& sensitive_properties, const
std::string& component_id, const std::string& property_name, const std::string&
property_value) {
+ const auto sensitive_property_it =
std::ranges::find_if(sensitive_properties, [&](const auto& sensitive_property) {
+return sensitive_property.component_id.to_string().view() == component_id
&& (sensitive_property.property_name == property_name ||
sensitive_property.property_display_name == property_name);
+ });
+ if (sensitive_property_it == sensitive_properties.end()) {
+std::cout << "No sensitive property found with this component ID and
property name.\n";
+return {};
+ }
+ return {{sensitive_property_it->component_id,
{{sensitive_property_it->property_name, property_value;
+}
+
+std::unordered_map> createOverridesForReEncryption(const
std::vector& sensitive_properties) {
+ std::unordered_map> overrides;
+ for (const auto& sensitive_property : sensitive_properties) {
+
overrides[sensitive_property.component_id].emplace(sensitive_property.property_name,
sensitive_property.property_value);
+ }
+ return overrides;
+}
+
+} // namespace
+
+namespace org::apache::nifi::minifi::encrypt_config::flow_config_encryptor {
+
+EncryptionRequest::EncryptionRequest(EncryptionType type) : type{type} {
+ gsl_Expects(type == EncryptionType::Interactive || type ==
EncryptionType::ReEncrypt);
+}
+
+EncryptionRequest::EncryptionRequest(std::string_view component_id,
std::string_view property_name, std::string_view property_value)
+: type{EncryptionType::SingleProperty},
+ component_id{component_id},
+ property_name{property_name},
+
[jira] [Commented] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823176#comment-17823176 ] David Handermann commented on NIFI-12858: - Thanks for describing this issue [~chriseconklin], it sounds like this was an unintended regression with the transition from H2 Database to JetBrains Xodus for storing and retrieving Flow Configuration History. I plan to address this soon. > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.25.0 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Attachments: nifi1.23.2.png, nifi1.24.png > > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12858: --- Assignee: David Handermann > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.25.0 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Attachments: nifi1.23.2.png, nifi1.24.png > > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
exceptionfactory commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511143065
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
Review Comment:
Thanks, I think in this case, the first version of this implementation could
hard-code `\n` instead of making it a property. If later on, there
Re: [PR] NIFI-12017 add ability to choose to output to single line for base32 base64 contents [nifi]
exceptionfactory commented on code in PR #8417:
URL: https://github.com/apache/nifi/pull/8417#discussion_r1511140439
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java:
##
@@ -29,55 +37,88 @@
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.encoding.EncodingMode;
+import org.apache.nifi.processors.standard.encoding.EncodingType;
+import org.apache.nifi.processors.standard.encoding.LineOutputMode;
import org.apache.nifi.processors.standard.util.ValidatingBase32InputStream;
import org.apache.nifi.processors.standard.util.ValidatingBase64InputStream;
import org.apache.nifi.stream.io.StreamUtils;
import org.apache.nifi.util.StopWatch;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
@SideEffectFree
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"encode", "decode", "base64", "base32", "hex"})
@CapabilityDescription("Encode or decode the contents of a FlowFile using
Base64, Base32, or hex encoding schemes")
public class EncodeContent extends AbstractProcessor {
-public static final String ENCODE_MODE = "Encode";
-public static final String DECODE_MODE = "Decode";
-
-public static final String BASE64_ENCODING = "base64";
-public static final String BASE32_ENCODING = "base32";
-public static final String HEX_ENCODING = "hex";
-
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
.description("Specifies whether the content should be encoded or
decoded")
.required(true)
-.allowableValues(ENCODE_MODE, DECODE_MODE)
-.defaultValue(ENCODE_MODE)
+.allowableValues(EncodingMode.class)
+.defaultValue(EncodingMode.ENCODE)
.build();
public static final PropertyDescriptor ENCODING = new
PropertyDescriptor.Builder()
.name("Encoding")
.description("Specifies the type of encoding used")
.required(true)
-.allowableValues(BASE64_ENCODING, BASE32_ENCODING, HEX_ENCODING)
-.defaultValue(BASE64_ENCODING)
+.allowableValues(EncodingType.class)
+.defaultValue(EncodingType.BASE64_ENCODING)
+.build();
+
+static final PropertyDescriptor LINE_OUTPUT_MODE = new
PropertyDescriptor.Builder()
+.name("Line Output Mode")
+.displayName("Line Output Mode")
+.description("If set to 'single-line', the encoded FlowFile
content will output as a single line. If set to 'multiple-lines', "
++ "it will output as multiple lines. This property is only
applicable when Base64 or Base32 encoding is selected.")
+.required(false)
+.defaultValue(LineOutputMode.SINGLE_LINE)
+.allowableValues(LineOutputMode.class)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
.build();
+static final PropertyDescriptor ENCODED_LINE_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Separator")
+.displayName("Encoded Content Line Separator")
+.description("Each line of encoded data will be terminated with this
byte sequence (e.g. \\r\\n"
++ "). This property defaults to the system-dependent line
separator string. If `line-length` <= 0, "
++ "the `line-separator` property is not used. This property is
not used for `hex` encoding.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.defaultValue(System.lineSeparator())
+.addValidator(Validator.VALID)
+.dependsOn(MODE, EncodingMode.ENCODE)
+.dependsOn(ENCODING, EncodingType.BASE64_ENCODING,
EncodingType.BASE32_ENCODING)
+.build();
+
+static final PropertyDescriptor ENCODED_LINE_LENGTH = new
PropertyDescriptor.Builder()
+.name("Encoded Content Line Length")
+
Re: [PR] NIFI-11107 In ConsumeIMAP and ConsumePOP3 added support for OAUTH based authorization. [nifi]
tpalfy commented on PR #6900: URL: https://github.com/apache/nifi/pull/6900#issuecomment-1976510039 @AnTu2702 The `oauth2AccessTokenProviderOptional` field is used exclusively in the `buildUrl` method where we call `oauth2AccessTokenProvider.getAccessDetails().getAccessToken()`. The `getAccessDetail()` should take care of refreshing/reacquiring tokens (this is the same call you suggest doing at the beginning of `onTrigger`). So according to this token expiration is handled already. Do we have proof that this doesn't work for some reason? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12859) OAUTH2 Token Refresh not working in AbstractEmailProcessor
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823131#comment-17823131 ] Andreas Tully commented on NIFI-12859: -- I debugged the AbstractEmailProcessor.java and it seems, that AbstractEmailProcessor.java does not handle token expiration at all. The StandardOAuth2AccessTokenProvider.java does, but it would need the AbstractEmailProcessor.java to call the: oauth2AccessTokenProvider.getAccessDetails() in the onTrigger() method instead the onSchedule() method to do so. [!https://private-user-images.githubusercontent.com/5054213/309678670-40403a5b-3a2f-4174-a1a2-af204650bd0e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDk1NTEwOTcsIm5iZiI6MTcwOTU1MDc5NywicGF0aCI6Ii81MDU0MjEzLzMwOTY3ODY3MC00MDQwM2E1Yi0zYTJmLTQxNzQtYTFhMi1hZjIwNDY1MGJkMGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDMwNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDAzMDRUMTExMzE3WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZDgwNmZiMDc4ZTNhZWYzOWI3YzE4NmNlZjE0MWI3ZmFlZmU0OWIxODdlMGE0M2JhN2EwYzc4NWJhMDk2MGJlMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.tFkjC7_5Bts96mIVPHYhSx61FC2cVwMK_gH9R8QDJUA!|https://private-user-images.githubusercontent.com/5054213/309678670-40403a5b-3a2f-4174-a1a2-af204650bd0e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDk1NTEwOTcsIm5iZiI6MTcwOTU1MDc5NywicGF0aCI6Ii81MDU0MjEzLzMwOTY3ODY3MC00MDQwM2E1Yi0zYTJmLTQxNzQtYTFhMi1hZjIwNDY1MGJkMGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDMwNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDAzMDRUMTExMzE3WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZDgwNmZiMDc4ZTNhZWYzOWI3YzE4NmNlZjE0MWI3ZmFlZmU0OWIxODdlMGE0M2JhN2EwYzc4NWJhMDk2MGJlMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.tFkjC7_5Bts96mIVPHYhSx61FC2cVwMK_gH9R8QDJUA] The AbstractEmailProcessor.java should be fixed to handle expired OAuth2 access tokens! > OAUTH2 Token Refresh not working in AbstractEmailProcessor > --- > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: image-2024-03-04-11-00-25-665.png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the ConsumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > onScheduled will be only triggered on start 1time. > onTriggered will be also triggered after start of the AbstractEmailProcessor. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > [https://github.com/apache/nifi/pull/6900] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2306 Filter out corrupt flowfiles during startup [nifi-minifi-cpp]
szaszm commented on code in PR #1738:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1738#discussion_r1510982246
##
extensions/rocksdb-repos/FlowFileRepository.cpp:
##
@@ -127,37 +142,37 @@ void FlowFileRepository::initialize_repository() {
}
logger_->log_info("Reading existing flow files from database");
- auto it = opendb->NewIterator(rocksdb::ReadOptions());
+ const auto it = opendb->NewIterator(rocksdb::ReadOptions());
for (it->SeekToFirst(); it->Valid(); it->Next()) {
utils::Identifier container_id;
auto eventRead =
FlowFileRecord::DeSerialize(gsl::make_span(it->value()).as_span(), content_repo_, container_id);
-std::string key = it->key().ToString();
-if (eventRead) {
- // on behalf of the just resurrected persisted instance
- auto claim = eventRead->getResourceClaim();
- if (claim) claim->increaseFlowFileRecordOwnedCount();
- bool found = false;
- auto search = containers_.find(container_id.to_string());
- found = (search != containers_.end());
- if (!found) {
-// for backward compatibility
-search = connection_map_.find(container_id.to_string());
-found = (search != connection_map_.end());
- }
- if (found) {
-logger_->log_debug("Found connection for {}, path {} ",
container_id.to_string(), eventRead->getContentFullPath());
-eventRead->setStoredToRepository(true);
-// we found the connection for the persistent flowFile
-// even if a processor immediately marks it for deletion, flush only
happens after prune_stored_flowfiles
-search->second->restore(eventRead);
- } else {
-logger_->log_warn("Could not find connection for {}, path {} ",
container_id.to_string(), eventRead->getContentFullPath());
-keys_to_delete_.enqueue({.key = key, .content =
eventRead->getResourceClaim()});
- }
-} else {
+const std::string key = it->key().ToString();
+if (!eventRead) {
// failed to deserialize FlowFile, cannot clear claim
keys_to_delete_.enqueue({.key = key});
+ continue;
+}
+auto claim = eventRead->getResourceClaim();
+if (claim) {
+ claim->increaseFlowFileRecordOwnedCount();
}
+const auto container = getContainer(container_id.to_string());
+if (!container) {
+ logger_->log_warn("Could not find connection for %s, path %s ",
container_id.to_string(), eventRead->getContentFullPath());
+ keys_to_delete_.enqueue({.key = key, .content =
eventRead->getResourceClaim()});
+ continue;
+}
+if (check_flowfile_content_size_ &&
!contentSizeIsAmpleForFlowFile(*eventRead, claim)) {
+ logger_->log_warn("Content size was insufficient for flowfile {}",
eventRead->getContentFullPath());
Review Comment:
I find this easier to understand.
```suggestion
logger_->log_warn("Content is missing or too small for flowfile {}",
eventRead->getContentFullPath());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-12860) Null exception from ExtensionMetadata caused by NIFI-12113
Peter Radden created NIFI-12860: --- Summary: Null exception from ExtensionMetadata caused by NIFI-12113 Key: NIFI-12860 URL: https://issues.apache.org/jira/browse/NIFI-12860 Project: Apache NiFi Issue Type: Bug Affects Versions: 2.0.0-M2, 1.25.0, 1.24.0 Reporter: Peter Radden Shown using command similar to: cli.sh registry list-extensions -ot json -p tk.properties ERROR: Error executing command 'list-extensions' : null https://github.com/apache/nifi/pull/8467 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12859) OAUTH2 Token Refresh not working in AbstractEmailProcessor
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Description: The ConsumeIMAP Processor implements the AbstractEmailProcessor. With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is working fine. But the refresh token implementation is not working with Credential Flow in OAUTH Processor. Reason: Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is called only in onScheduled Method of AbstractEmailProcessor. But once the ConsumeIMAPProcessor is startet (also for cron based scheduling) only the onTriggeredMethod will be triggered. onScheduled will be only triggered on start 1time. onTriggered will be also triggered after start of the AbstractEmailProcessor. Abstract-Email-Processor: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] Here you find details: [https://github.com/apache/nifi/pull/6900] was: The ConsumeIMAP Processor implements the AbstractEmailProcessor. With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is working fine. But the refresh token implementation is not working with Credential Flow in OAUTH Processor. Reason: Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is called only in onScheduled Method of AbstractEmailProcessor. But once the consumeIMAPProcessor is startet (also for cron based scheduling) only the onTriggeredMethod will be triggered. OnScheduled will be only triggered on start 1time. OnTriggered will be also triggered after start of the consumeImapProcessor. So I would suggest to move the parts of code for oauth2AccessTokenProvider from onScheduledMethod to onTriggeredMethod. Abstract-Email-Processor: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] Here you find details: https://github.com/apache/nifi/pull/6900 Summary: OAUTH2 Token Refresh not working in AbstractEmailProcessor (was: OAUTH Refresh not working in ConsumeIMAP ) > OAUTH2 Token Refresh not working in AbstractEmailProcessor > --- > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: image-2024-03-04-11-00-25-665.png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the ConsumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > onScheduled will be only triggered on start 1time. > onTriggered will be also triggered after start of the AbstractEmailProcessor. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > [https://github.com/apache/nifi/pull/6900] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Attachment: (was: Bild (1).png) > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: image-2024-03-04-11-00-25-665.png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823114#comment-17823114 ] Roman commented on NIFI-12859: -- Code-Screenshot how it could be solved. At the moment no permissions to push a MR !image-2024-03-04-11-00-25-665.png|width=1017,height=287! > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: Bild (1).png, image-2024-03-04-11-00-25-665.png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Attachment: image-2024-03-04-11-00-25-665.png > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: Bild (1).png, image-2024-03-04-11-00-25-665.png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Attachment: Bild (1).png > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > Attachments: Bild (1).png > > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11107 In ConsumeIMAP and ConsumePOP3 added support for OAUTH based authorization. [nifi]
AnTu2702 commented on PR #6900: URL: https://github.com/apache/nifi/pull/6900#issuecomment-1976176344 Here would be my suggestion for a fix:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-2306) Filter out corrupt flowfiles during startup
[ https://issues.apache.org/jira/browse/MINIFICPP-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2306: --- Status: Patch Available (was: In Progress) https://github.com/apache/nifi-minifi-cpp/pull/1738 > Filter out corrupt flowfiles during startup > --- > > Key: MINIFICPP-2306 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2306 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes user's encounter flowfiles where the content is corrupt this can > clog up various processors. This is usually handled in the processor but it > would be nice to optionally throw these flowfiles out during startup. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] Null exception from ExtensionMetadata caused by NIFI-12113 [nifi]
pradden opened a new pull request, #8467: URL: https://github.com/apache/nifi/pull/8467 This issue is present in all branches from version 1.24 onwards -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Priority: Major (was: Minor) > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Major > Labels: OAuth2 > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
[ https://issues.apache.org/jira/browse/NIFI-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman updated NIFI-12859: - Description: The ConsumeIMAP Processor implements the AbstractEmailProcessor. With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is working fine. But the refresh token implementation is not working with Credential Flow in OAUTH Processor. Reason: Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is called only in onScheduled Method of AbstractEmailProcessor. But once the consumeIMAPProcessor is startet (also for cron based scheduling) only the onTriggeredMethod will be triggered. OnScheduled will be only triggered on start 1time. OnTriggered will be also triggered after start of the consumeImapProcessor. So I would suggest to move the parts of code for oauth2AccessTokenProvider from onScheduledMethod to onTriggeredMethod. Abstract-Email-Processor: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] Here you find details: https://github.com/apache/nifi/pull/6900 was: The ConsumeIMAP Processor implements the AbstractEmailProcessor. With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is working fine. But the refresh token implementation is not working with Credential Flow in OAUTH Processor. Reason: Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is called only in onScheduled Method of AbstractEmailProcessor. But once the consumeIMAPProcessor is startet (also for cron based scheduling) only the onTriggeredMethod will be triggered. OnScheduled will be only triggered on start 1time. OnTriggered will be also triggered after start of the consumeImapProcessor. So I would suggest to move the parts of code for oauth2AccessTokenProvider from onScheduledMethod to onTriggeredMethod > OAUTH Refresh not working in ConsumeIMAP > - > > Key: NIFI-12859 > URL: https://issues.apache.org/jira/browse/NIFI-12859 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0, 1.25.0 >Reporter: Roman >Priority: Minor > Labels: OAuth2 > > The ConsumeIMAP Processor implements the AbstractEmailProcessor. > With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is > working fine. > But the refresh token implementation is not working with Credential Flow in > OAUTH Processor. > Reason: > Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is > called only in onScheduled Method of AbstractEmailProcessor. > But once the consumeIMAPProcessor is startet (also for cron based scheduling) > only the onTriggeredMethod will be triggered. > OnScheduled will be only triggered on start 1time. > OnTriggered will be also triggered after start of the consumeImapProcessor. > So I would suggest to move the parts of code for oauth2AccessTokenProvider > from onScheduledMethod to onTriggeredMethod. > Abstract-Email-Processor: > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java] > Here you find details: > https://github.com/apache/nifi/pull/6900 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12859) OAUTH Refresh not working in ConsumeIMAP
Roman created NIFI-12859: Summary: OAUTH Refresh not working in ConsumeIMAP Key: NIFI-12859 URL: https://issues.apache.org/jira/browse/NIFI-12859 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.25.0, 1.24.0 Reporter: Roman The ConsumeIMAP Processor implements the AbstractEmailProcessor. With Issue NIFI-11107 there was introduced OAUTH2 Support. Initial OAUTH is working fine. But the refresh token implementation is not working with Credential Flow in OAUTH Processor. Reason: Method to refresh token => oauth2AccessTokenProvider.getAccessDetails(); is called only in onScheduled Method of AbstractEmailProcessor. But once the consumeIMAPProcessor is startet (also for cron based scheduling) only the onTriggeredMethod will be triggered. OnScheduled will be only triggered on start 1time. OnTriggered will be also triggered after start of the consumeImapProcessor. So I would suggest to move the parts of code for oauth2AccessTokenProvider from onScheduledMethod to onTriggeredMethod -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11107 In ConsumeIMAP and ConsumePOP3 added support for OAUTH based authorization. [nifi]
AnTu2702 commented on PR #6900: URL: https://github.com/apache/nifi/pull/6900#issuecomment-1975971033 I debugged the AbstractEmailProcessor.java and it seems, that AbstractEmailProcessor.java does not handle token expiration at all. The StandardOAuth2AccessTokenProvider.Java does, but it would need the AbstractEmailProcessor.java to call the: oauth2AccessTokenProvider.getAccessDetails() in the onTrigger() method instead the onSchedule() method to do so.  The AbstractEmailProcessor.java should be fixed to handle expired OAuth2 access tokens! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
