[jira] [Commented] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057600#comment-17057600 ] Shreyas KC commented on NIFI-7246: -- [~pvillard] , [~alopresto] I understand it is by design, This implementation is designed to override the behaviour if the admin chooses to, as it is config driven with default behaviour to be the older one. As NiFi is moving to docker/Kubernetes ecosystem, it is primordial we start designing idempotency in the software design and this is a good start. > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Security >Reporter: Shreyas KC >Priority: Major > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7241) Unable to remove port from a child ProcessGroup
[ https://issues.apache.org/jira/browse/NIFI-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-7241: --- Fix Version/s: 1.11.4 1.12.0 > Unable to remove port from a child ProcessGroup > --- > > Key: NIFI-7241 > URL: https://issues.apache.org/jira/browse/NIFI-7241 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Reporter: Wouter de Vries >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0, 1.11.4 > > Attachments: trace.log > > Time Spent: 10m > Remaining Estimate: 0h > > Given the following scenario: > * Output port 1 in process group A > * Input port 2 in process group B > * A connection between A and B > * Process group C is versioned and contains A and B > Removing the connection as well as output port 1 and then committing the > version creates a flow version that cannot be upgraded to. (Try reverting to > an older version, and back to this version). Throwing an > IllegalStateException. > I believe this is caused by the order of updates in > "StandardProcessGroup.java" where child groups (line 3781) are updated prior > to the connections of the group itself (line 3986). This causes the output > port to be removed before the connection from it is removed. I haven't tested > this but probably removing the input port causes similar issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7119) Implement boundary checking for Argon2 cost parameters
[
https://issues.apache.org/jira/browse/NIFI-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7119:
Fix Version/s: 1.12.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Implement boundary checking for Argon2 cost parameters
> --
>
> Key: NIFI-7119
> URL: https://issues.apache.org/jira/browse/NIFI-7119
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Major
> Labels: beginner, boundary, hashing, security, validation
> Fix For: 1.12.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{Argon2}} secure hasher added for the flow fingerprint fix does not
> enforce boundaries around the cost parameters provided to the various
> constructors. It should restrict provided values to valid entries as noted in
> the Javadoc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] alopresto merged pull request #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters
alopresto merged pull request #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters URL: https://github.com/apache/nifi/pull/4111 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-7119) Implement boundary checking for Argon2 cost parameters
[
https://issues.apache.org/jira/browse/NIFI-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057454#comment-17057454
]
ASF subversion and git services commented on NIFI-7119:
---
Commit 290bd378d5e219dabac8f3ecf2bf9c69451f1c3c in nifi's branch
refs/heads/master from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=290bd37 ]
NIFI-7119 Implement boundary checking for Argon2 cost parameters (#4111)
* NIFI-7119 Implemented parameter boundary enforcement for Argon2SecureHasher
constructor.
Added unit tests for validating each parameter check.
* NIFI-7119 Refactored parameter validations. Added more test sizes to boundary
checkers. Changed logger severity to error and added bounds to messages.
* NIFI-7119 Refactored Argon2 parameter data types to handle unsigned integer
boundary values.
Updated unit tests.
Co-authored-by: Andy LoPresto
Signed-off-by: Andy LoPresto
> Implement boundary checking for Argon2 cost parameters
> --
>
> Key: NIFI-7119
> URL: https://issues.apache.org/jira/browse/NIFI-7119
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Major
> Labels: beginner, boundary, hashing, security, validation
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{Argon2}} secure hasher added for the flow fingerprint fix does not
> enforce boundaries around the cost parameters provided to the various
> constructors. It should restrict provided values to valid entries as noted in
> the Javadoc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7119) Implement boundary checking for Argon2 cost parameters
[
https://issues.apache.org/jira/browse/NIFI-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057456#comment-17057456
]
ASF subversion and git services commented on NIFI-7119:
---
Commit 290bd378d5e219dabac8f3ecf2bf9c69451f1c3c in nifi's branch
refs/heads/master from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=290bd37 ]
NIFI-7119 Implement boundary checking for Argon2 cost parameters (#4111)
* NIFI-7119 Implemented parameter boundary enforcement for Argon2SecureHasher
constructor.
Added unit tests for validating each parameter check.
* NIFI-7119 Refactored parameter validations. Added more test sizes to boundary
checkers. Changed logger severity to error and added bounds to messages.
* NIFI-7119 Refactored Argon2 parameter data types to handle unsigned integer
boundary values.
Updated unit tests.
Co-authored-by: Andy LoPresto
Signed-off-by: Andy LoPresto
> Implement boundary checking for Argon2 cost parameters
> --
>
> Key: NIFI-7119
> URL: https://issues.apache.org/jira/browse/NIFI-7119
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Major
> Labels: beginner, boundary, hashing, security, validation
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{Argon2}} secure hasher added for the flow fingerprint fix does not
> enforce boundaries around the cost parameters provided to the various
> constructors. It should restrict provided values to valid entries as noted in
> the Javadoc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] alopresto commented on issue #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters
alopresto commented on issue #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters URL: https://github.com/apache/nifi/pull/4111#issuecomment-597918991 Ran `contrib-check` and all tests pass. +1, merging. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-7119) Implement boundary checking for Argon2 cost parameters
[
https://issues.apache.org/jira/browse/NIFI-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057455#comment-17057455
]
ASF subversion and git services commented on NIFI-7119:
---
Commit 290bd378d5e219dabac8f3ecf2bf9c69451f1c3c in nifi's branch
refs/heads/master from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=290bd37 ]
NIFI-7119 Implement boundary checking for Argon2 cost parameters (#4111)
* NIFI-7119 Implemented parameter boundary enforcement for Argon2SecureHasher
constructor.
Added unit tests for validating each parameter check.
* NIFI-7119 Refactored parameter validations. Added more test sizes to boundary
checkers. Changed logger severity to error and added bounds to messages.
* NIFI-7119 Refactored Argon2 parameter data types to handle unsigned integer
boundary values.
Updated unit tests.
Co-authored-by: Andy LoPresto
Signed-off-by: Andy LoPresto
> Implement boundary checking for Argon2 cost parameters
> --
>
> Key: NIFI-7119
> URL: https://issues.apache.org/jira/browse/NIFI-7119
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Major
> Labels: beginner, boundary, hashing, security, validation
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{Argon2}} secure hasher added for the flow fingerprint fix does not
> enforce boundaries around the cost parameters provided to the various
> constructors. It should restrict provided values to valid entries as noted in
> the Javadoc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7119) Implement boundary checking for Argon2 cost parameters
[
https://issues.apache.org/jira/browse/NIFI-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057457#comment-17057457
]
ASF subversion and git services commented on NIFI-7119:
---
Commit 290bd378d5e219dabac8f3ecf2bf9c69451f1c3c in nifi's branch
refs/heads/master from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=290bd37 ]
NIFI-7119 Implement boundary checking for Argon2 cost parameters (#4111)
* NIFI-7119 Implemented parameter boundary enforcement for Argon2SecureHasher
constructor.
Added unit tests for validating each parameter check.
* NIFI-7119 Refactored parameter validations. Added more test sizes to boundary
checkers. Changed logger severity to error and added bounds to messages.
* NIFI-7119 Refactored Argon2 parameter data types to handle unsigned integer
boundary values.
Updated unit tests.
Co-authored-by: Andy LoPresto
Signed-off-by: Andy LoPresto
> Implement boundary checking for Argon2 cost parameters
> --
>
> Key: NIFI-7119
> URL: https://issues.apache.org/jira/browse/NIFI-7119
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Major
> Labels: beginner, boundary, hashing, security, validation
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{Argon2}} secure hasher added for the flow fingerprint fix does not
> enforce boundaries around the cost parameters provided to the various
> constructors. It should restrict provided values to valid entries as noted in
> the Javadoc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] alopresto commented on a change in pull request #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters
alopresto commented on a change in pull request #4111: NIFI-7119 Implement boundary checking for Argon2 cost parameters URL: https://github.com/apache/nifi/pull/4111#discussion_r391314412 ## File path: nifi-commons/nifi-security-utils/src/main/java/org/apache/nifi/security/util/crypto/Argon2SecureHasher.java ## @@ -53,11 +53,14 @@ private final int iterations; private final int saltLength; -private final boolean usingStaticSalt; +private boolean usingStaticSalt; // A 16 byte salt (nonce) is recommended for password hashing private static final byte[] staticSalt = "NiFi Static Salt".getBytes(StandardCharsets.UTF_8); +// Upper boundary for several cost parameters +private static final double upperBoundary = Math.pow(2, 32) - 1; Review comment: Used `Integer` to hold unsigned ints. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] pvillard31 commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2
pvillard31 commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2 URL: https://github.com/apache/nifi/pull/4126#issuecomment-597917009 Sorry I just read again your previous answer... you want to add it as a future addition. Please disregard my comments then ;) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] pvillard31 commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2
pvillard31 commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2 URL: https://github.com/apache/nifi/pull/4126#issuecomment-597916659 Yes I was talking about CMEK. Totally fine with a follow-up JIRA. Regarding the controller service, I don't see it as an option in this new processor. Here is the configuration for the Blob processor: https://user-images.githubusercontent.com/11541012/76471064-fb251100-63f1-11ea-8096-962210554e67.png;> Where Storage Credentials is the possible reference to the controller service. CS that I could configure with: https://user-images.githubusercontent.com/11541012/76471099-1263fe80-63f2-11ea-985b-99cd0e8cc97a.png;> With this new processor, I don't see the Controller Service option: https://user-images.githubusercontent.com/11541012/76470968-bf8a4700-63f1-11ea-9f91-0d80a36033a4.png;> Am I missing something? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] taftster opened a new pull request #4137: NIFI-6742 Use JUnit TemporaryFolder when creating test databases
taftster opened a new pull request #4137: NIFI-6742 Use JUnit TemporaryFolder when creating test databases URL: https://github.com/apache/nifi/pull/4137 - Add @Rule for TemporaryFolder - Replace use of previous target/db with TemporaryFolder/db - Remove use of ~/test db (in home directory) - Remove System.out lines In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] MuazmaZ commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2
MuazmaZ commented on issue #4126: NIFI-7103 Adding PutAzureDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2 URL: https://github.com/apache/nifi/pull/4126#issuecomment-597898985 @pvillard31 thanks for the feedback. - ADLS by default is encrypted with Microsoft Managed Keys. We can definitely add Key ID/Key Vault functionality for Customer Managed Keys for encryption as a feature add. - Following Azure Blob Storage pattern we want to provide both options with Controller Service and with properties. I plan to add it along with permission (ACLs) as future additions. Let me know if it clarifies your questions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7246: - Issue Type: Improvement (was: Bug) > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Shreyas KC >Priority: Major > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7246: - Component/s: Security > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Security >Reporter: Shreyas KC >Priority: Major > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7246: - Fix Version/s: (was: 1.11.3) > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Shreyas KC >Priority: Major > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7246: - Affects Version/s: (was: 1.11.3) > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Shreyas KC >Priority: Major > Fix For: 1.11.3 > > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
[ https://issues.apache.org/jira/browse/NIFI-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057399#comment-17057399 ] Andy LoPresto commented on NIFI-7246: - The uniqueness of JWT-generating keys to individual nodes (and thus the lack of transitivity of tokens between nodes) is by design. The solution is to enable sticky sessions in the load balancer. > JWT Generated by a node in the cluster is not honored by other nodes in the > cluster. > > > Key: NIFI-7246 > URL: https://issues.apache.org/jira/browse/NIFI-7246 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.11.3 >Reporter: Shreyas KC >Priority: Major > Fix For: 1.11.3 > > > In an externally load balanced cluster without sticky session, it is not > possible to currently share the JWT generated by one node with the rest of > the nodes in the cluster. > Hence we need a mechanism where we can introduce static key in the > nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key
Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597877840 > there appears to be several conflicts to resolve. there also needs to be a JIRA that this links to Added a JIRA : https://issues.apache.org/jira/browse/NIFI-7246 and resolved conflicts. I have deployed this fix in my organization and its working fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] Shreyaskc removed a comment on issue #4135: External load balancer support for JWT using static, configured key
Shreyaskc removed a comment on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597877075 https://issues.apache.org/jira/browse/NIFI-7246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] Shreyaskc removed a comment on issue #4135: External load balancer support for JWT using static, configured key
Shreyaskc removed a comment on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597874674 Conflicts resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-7246) JWT Generated by a node in the cluster is not honored by other nodes in the cluster.
Shreyas KC created NIFI-7246: Summary: JWT Generated by a node in the cluster is not honored by other nodes in the cluster. Key: NIFI-7246 URL: https://issues.apache.org/jira/browse/NIFI-7246 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.11.3 Reporter: Shreyas KC Fix For: 1.11.3 In an externally load balanced cluster without sticky session, it is not possible to currently share the JWT generated by one node with the rest of the nodes in the cluster. Hence we need a mechanism where we can introduce static key in the nifi.properties in its chosen by the cluster administrator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key
Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597877075 https://issues.apache.org/jira/browse/NIFI-7246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key
Shreyaskc commented on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597874674 Conflicts resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7241) Unable to remove port from a child ProcessGroup
[ https://issues.apache.org/jira/browse/NIFI-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7241: - Affects Version/s: (was: 1.11.2) > Unable to remove port from a child ProcessGroup > --- > > Key: NIFI-7241 > URL: https://issues.apache.org/jira/browse/NIFI-7241 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Reporter: Wouter de Vries >Assignee: Mark Payne >Priority: Major > Attachments: trace.log > > Time Spent: 10m > Remaining Estimate: 0h > > Given the following scenario: > * Output port 1 in process group A > * Input port 2 in process group B > * A connection between A and B > * Process group C is versioned and contains A and B > Removing the connection as well as output port 1 and then committing the > version creates a flow version that cannot be upgraded to. (Try reverting to > an older version, and back to this version). Throwing an > IllegalStateException. > I believe this is caused by the order of updates in > "StandardProcessGroup.java" where child groups (line 3781) are updated prior > to the connections of the group itself (line 3986). This causes the output > port to be removed before the connection from it is removed. I haven't tested > this but probably removing the input port causes similar issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7241) Unable to remove port from a child ProcessGroup
[ https://issues.apache.org/jira/browse/NIFI-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7241: - Assignee: Mark Payne Status: Patch Available (was: Open) > Unable to remove port from a child ProcessGroup > --- > > Key: NIFI-7241 > URL: https://issues.apache.org/jira/browse/NIFI-7241 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.11.2 >Reporter: Wouter de Vries >Assignee: Mark Payne >Priority: Major > Attachments: trace.log > > Time Spent: 10m > Remaining Estimate: 0h > > Given the following scenario: > * Output port 1 in process group A > * Input port 2 in process group B > * A connection between A and B > * Process group C is versioned and contains A and B > Removing the connection as well as output port 1 and then committing the > version creates a flow version that cannot be upgraded to. (Try reverting to > an older version, and back to this version). Throwing an > IllegalStateException. > I believe this is caused by the order of updates in > "StandardProcessGroup.java" where child groups (line 3781) are updated prior > to the connections of the group itself (line 3986). This causes the output > port to be removed before the connection from it is removed. I haven't tested > this but probably removing the input port causes similar issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #4136: NIFI-7241: When updating Process Group to match VersionedProcessGroup…
markap14 opened a new pull request #4136: NIFI-7241: When updating Process
Group to match VersionedProcessGroup…
URL: https://github.com/apache/nifi/pull/4136
…, remove any connections before recursing into child groups. This ensures
that if a Port exists in child group A and is connected to a port in child
group B, if the VersionedProcessGroup indicates to remove the port, that
connection will be removed before attempting to remove the port. Updating and
adding connections must still be done last, after all components have been
added. But missing connections can be removed earlier.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] joewitt commented on issue #4135: External load balancer support for JWT using static, configured key
joewitt commented on issue #4135: External load balancer support for JWT using static, configured key URL: https://github.com/apache/nifi/pull/4135#issuecomment-597858937 there appears to be several conflicts to resolve. there also needs to be a JIRA that this links to This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] Shreyaskc opened a new pull request #4135: External load balancer support for JWT using static, configured key
Shreyaskc opened a new pull request #4135: External load balancer support for
JWT using static, configured key
URL: https://github.com/apache/nifi/pull/4135
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
[ https://issues.apache.org/jira/browse/NIFI-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-7245: --- Fix Version/s: 1.11.4 > JMS processors should yield when connection factory initialisation failed > - > > Key: NIFI-7245 > URL: https://issues.apache.org/jira/browse/NIFI-7245 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Fix For: 1.12.0, 1.11.4 > > Time Spent: 20m > Remaining Estimate: 0h > > JMS processors yield properly when there is an exception during > getConnection() and the steps after (get session, send / receive messages). > They should also yield if the connection factory initialisation is > unsuccessful (eg. due to missing JMS client libraries or misconfiguration of > the factory class name). Otherwise the processors produce a lot of logs > continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7242) Parameters update not taken into account in controller services
[
https://issues.apache.org/jira/browse/NIFI-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-7242.
--
Resolution: Fixed
> Parameters update not taken into account in controller services
> ---
>
> Key: NIFI-7242
> URL: https://issues.apache.org/jira/browse/NIFI-7242
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.12.0, 1.11.4
>
> Attachments: parametersIssue.xml
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> There is a bug with the parameters when used in Controller Services:
> * when updating a parameter that is referenced in a controller service (in
> this case avro schema registry), changing the value of the parameter does not
> seem to trigger the restart of the controller service
> * even if I do restart the components manually, the old value of the
> parameter is still used... NiFi restart is the only way to get the new value
> applied
> With the supplied template, create a Parameter Context with schema =
> {code:java}
> {
> "type" : "record",
> "name" : "myData",
> "namespace" : "myLine",
> "fields" : [
> {
> "name" : "myField1",
> "type" : "string"
> }
> ]
> }
> {code}
> The AvroSchemaRegistry contains the schema with:
> schema => #\{schema}
> Get everything running: output data has only one column. Then update the
> Parameter Context to have schema =
> {code:java}
> {
> "type" : "record",
> "name" : "myData",
> "namespace" : "myLine",
> "fields" : [
> {
> "name" : "myField1",
> "type" : "string"
> }, {
> "name" : "myField2",
> "type" : "string"
> }
> ]
> }
> {code}
> Output data has still one column only when it should have two with the new
> schema.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7242) Parameters update not taken into account in controller services
[
https://issues.apache.org/jira/browse/NIFI-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057350#comment-17057350
]
ASF subversion and git services commented on NIFI-7242:
---
Commit d68720920fa10155eff81e1c1314a9286ba7a51c in nifi's branch
refs/heads/master from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d687209 ]
NIFI-7242: When a Parameter is changed, any property referencing that parameter
should have its #onPropertyModified method called. Also renamed Accumulo tests
to integration tests because they start embedded servers and connect to them,
which caused failures in my environment. Also fixed a bug in
TestLengthDelimitedJournal because it was resulting in failures when building
locally as well.
Signed-off-by: Pierre Villard
This closes #4134.
> Parameters update not taken into account in controller services
> ---
>
> Key: NIFI-7242
> URL: https://issues.apache.org/jira/browse/NIFI-7242
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.12.0, 1.11.4
>
> Attachments: parametersIssue.xml
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> There is a bug with the parameters when used in Controller Services:
> * when updating a parameter that is referenced in a controller service (in
> this case avro schema registry), changing the value of the parameter does not
> seem to trigger the restart of the controller service
> * even if I do restart the components manually, the old value of the
> parameter is still used... NiFi restart is the only way to get the new value
> applied
> With the supplied template, create a Parameter Context with schema =
> {code:java}
> {
> "type" : "record",
> "name" : "myData",
> "namespace" : "myLine",
> "fields" : [
> {
> "name" : "myField1",
> "type" : "string"
> }
> ]
> }
> {code}
> The AvroSchemaRegistry contains the schema with:
> schema => #\{schema}
> Get everything running: output data has only one column. Then update the
> Parameter Context to have schema =
> {code:java}
> {
> "type" : "record",
> "name" : "myData",
> "namespace" : "myLine",
> "fields" : [
> {
> "name" : "myField1",
> "type" : "string"
> }, {
> "name" : "myField2",
> "type" : "string"
> }
> ]
> }
> {code}
> Output data has still one column only when it should have two with the new
> schema.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
asfgit closed pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that… URL: https://github.com/apache/nifi/pull/4134 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] pvillard31 commented on issue #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
pvillard31 commented on issue #4134: NIFI-7242: When a Parameter is changed, any property referencing that… URL: https://github.com/apache/nifi/pull/4134#issuecomment-597836252 Code change looks good to me, builds are green, I tested the case I initially reported in the JIRA and this now works as expected. Merging to master. Thanks @markap14 ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFIREG-370) StandardRevisableEntityService returns wrong version on update
[ https://issues.apache.org/jira/browse/NIFIREG-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFIREG-370: Priority: Minor (was: Major) > StandardRevisableEntityService returns wrong version on update > -- > > Key: NIFIREG-370 > URL: https://issues.apache.org/jira/browse/NIFIREG-370 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The StandardRevisableEntityService converges create and update into a shared > method that returns a revision where the version is the incremented version > from the request. In some cases this is fine, but in the case where the > update was allowed because the client id was the same, it won't return > correct version which could be different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] bbende opened a new pull request #266: NIFIREG-370 Ensuring update retrieves the latest revision from Revisi…
bbende opened a new pull request #266: NIFIREG-370 Ensuring update retrieves the latest revision from Revisi… URL: https://github.com/apache/nifi-registry/pull/266 …onManager This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFIREG-370) StandardRevisableEntityService returns wrong version on update
Bryan Bende created NIFIREG-370: --- Summary: StandardRevisableEntityService returns wrong version on update Key: NIFIREG-370 URL: https://issues.apache.org/jira/browse/NIFIREG-370 Project: NiFi Registry Issue Type: Bug Affects Versions: 0.5.0 Reporter: Bryan Bende Assignee: Bryan Bende Fix For: 1.0.0 The StandardRevisableEntityService converges create and update into a shared method that returns a revision where the version is the incremented version from the request. In some cases this is fine, but in the case where the update was allowed because the client id was the same, it won't return correct version which could be different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] guvencenanguvenal commented on issue #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
guvencenanguvenal commented on issue #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC… URL: https://github.com/apache/nifi/pull/4131#issuecomment-597818176 I use Apache Nifi for so long time but I am new on Apache Nifi source code. Thanks your edited but i want to read flowfile attribute so i close PR :) i will work on it again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] guvencenanguvenal closed pull request #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
guvencenanguvenal closed pull request #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC… URL: https://github.com/apache/nifi/pull/4131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7236) Fix MAP handling in JsonRowRecordReader
[ https://issues.apache.org/jira/browse/NIFI-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7236: - Fix Version/s: 1.11.4 > Fix MAP handling in JsonRowRecordReader > --- > > Key: NIFI-7236 > URL: https://issues.apache.org/jira/browse/NIFI-7236 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.12.0, 1.11.4 > > Time Spent: 20m > Remaining Estimate: 0h > > Fixing a bug in JsonRowRecordReader when reading a 'raw' record with a schema > that indicates that a field should be a Map. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] pvillard31 commented on a change in pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
pvillard31 commented on a change in pull request #4134: NIFI-7242: When a
Parameter is changed, any property referencing that…
URL: https://github.com/apache/nifi/pull/4134#discussion_r391184601
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractComponentNode.java
##
@@ -815,12 +818,110 @@ public PropertyDescriptor getPropertyDescriptor(final
String name) {
}
-private void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+protected void onPropertyModified(final PropertyDescriptor descriptor,
final String oldValue, final String newValue) {
try (final NarCloseable narCloseable =
NarCloseable.withComponentNarLoader(extensionManager,
getComponent().getClass(), getComponent().getIdentifier())) {
getComponent().onPropertyModified(descriptor, oldValue, newValue);
}
}
+@Override
+public void onParametersModified(final Map
updatedParameters) {
+// If the component doesn't reference any parameters, then there's
nothing to be done.
+if (!isReferencingParameter()) {
+return;
+}
+
+final ParameterLookup previousParameterLookup =
createParameterLookupForPreviousValues(updatedParameters);
+
+// For any Property that references an updated Parameter, we need to
call onPropertyModified().
+// Additionally, we need to trigger validation to run if this
component is affected by the parameter update.
+boolean componentAffected = false;
+for (final Map.Entry entry
: properties.entrySet()) {
+final PropertyDescriptor propertyDescriptor = entry.getKey();
+final PropertyConfiguration configuration = entry.getValue();
+
+// Determine if this property is affected by the Parameter Update
+boolean propertyAffected = false;
+final List parameterReferences =
configuration.getParameterReferences();
+for (final ParameterReference reference : parameterReferences) {
+final String referencedParamName =
reference.getParameterName();
+if (updatedParameters.containsKey(referencedParamName)) {
+propertyAffected = true;
+componentAffected = true;
+break;
+}
+}
+
+if (propertyAffected) {
+final String previousValue =
configuration.getEffectiveValue(previousParameterLookup);
+final String updatedValue =
configuration.getEffectiveValue(getParameterContext());
+
+// Check if the value of the property is truly affected. It's
possible that we could have a property configured as something like "#{a}#{b}"
+// Where parameter a = "abc-" and b = "cba". The update could
change a to "abc" and b to "-cba". As a result, the property value previously
was "abc-cba" and still is.
+// In such a case, we should not call onPropertyModified.
+final boolean propertyUpdated = !Objects.equals(previousValue,
updatedValue);
+if (propertyUpdated) {
+try {
+logger.debug("Parameter Context updated, resulting in
property {} of {} changing. Calling onPropertyModified().", propertyDescriptor,
this);
Review comment:
You're right... I looked at it too quickly and assumed that the line below
was containing the parameters of the log statement. Doh.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] markap14 commented on a change in pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
markap14 commented on a change in pull request #4134: NIFI-7242: When a
Parameter is changed, any property referencing that…
URL: https://github.com/apache/nifi/pull/4134#discussion_r391181945
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractComponentNode.java
##
@@ -815,12 +818,110 @@ public PropertyDescriptor getPropertyDescriptor(final
String name) {
}
-private void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+protected void onPropertyModified(final PropertyDescriptor descriptor,
final String oldValue, final String newValue) {
try (final NarCloseable narCloseable =
NarCloseable.withComponentNarLoader(extensionManager,
getComponent().getClass(), getComponent().getIdentifier())) {
getComponent().onPropertyModified(descriptor, oldValue, newValue);
}
}
+@Override
+public void onParametersModified(final Map
updatedParameters) {
+// If the component doesn't reference any parameters, then there's
nothing to be done.
+if (!isReferencingParameter()) {
+return;
+}
+
+final ParameterLookup previousParameterLookup =
createParameterLookupForPreviousValues(updatedParameters);
+
+// For any Property that references an updated Parameter, we need to
call onPropertyModified().
+// Additionally, we need to trigger validation to run if this
component is affected by the parameter update.
+boolean componentAffected = false;
+for (final Map.Entry entry
: properties.entrySet()) {
+final PropertyDescriptor propertyDescriptor = entry.getKey();
+final PropertyConfiguration configuration = entry.getValue();
+
+// Determine if this property is affected by the Parameter Update
+boolean propertyAffected = false;
+final List parameterReferences =
configuration.getParameterReferences();
+for (final ParameterReference reference : parameterReferences) {
+final String referencedParamName =
reference.getParameterName();
+if (updatedParameters.containsKey(referencedParamName)) {
+propertyAffected = true;
+componentAffected = true;
+break;
+}
+}
+
+if (propertyAffected) {
+final String previousValue =
configuration.getEffectiveValue(previousParameterLookup);
+final String updatedValue =
configuration.getEffectiveValue(getParameterContext());
+
+// Check if the value of the property is truly affected. It's
possible that we could have a property configured as something like "#{a}#{b}"
+// Where parameter a = "abc-" and b = "cba". The update could
change a to "abc" and b to "-cba". As a result, the property value previously
was "abc-cba" and still is.
+// In such a case, we should not call onPropertyModified.
+final boolean propertyUpdated = !Objects.equals(previousValue,
updatedValue);
+if (propertyUpdated) {
+try {
+logger.debug("Parameter Context updated, resulting in
property {} of {} changing. Calling onPropertyModified().", propertyDescriptor,
this);
Review comment:
This is fine to log if it's sensitive. We're not logging the value, just the
descriptor.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] pvillard31 commented on a change in pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
pvillard31 commented on a change in pull request #4134: NIFI-7242: When a
Parameter is changed, any property referencing that…
URL: https://github.com/apache/nifi/pull/4134#discussion_r391179441
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractComponentNode.java
##
@@ -815,12 +818,110 @@ public PropertyDescriptor getPropertyDescriptor(final
String name) {
}
-private void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+protected void onPropertyModified(final PropertyDescriptor descriptor,
final String oldValue, final String newValue) {
try (final NarCloseable narCloseable =
NarCloseable.withComponentNarLoader(extensionManager,
getComponent().getClass(), getComponent().getIdentifier())) {
getComponent().onPropertyModified(descriptor, oldValue, newValue);
}
}
+@Override
+public void onParametersModified(final Map
updatedParameters) {
+// If the component doesn't reference any parameters, then there's
nothing to be done.
+if (!isReferencingParameter()) {
+return;
+}
+
+final ParameterLookup previousParameterLookup =
createParameterLookupForPreviousValues(updatedParameters);
+
+// For any Property that references an updated Parameter, we need to
call onPropertyModified().
+// Additionally, we need to trigger validation to run if this
component is affected by the parameter update.
+boolean componentAffected = false;
+for (final Map.Entry entry
: properties.entrySet()) {
+final PropertyDescriptor propertyDescriptor = entry.getKey();
+final PropertyConfiguration configuration = entry.getValue();
+
+// Determine if this property is affected by the Parameter Update
+boolean propertyAffected = false;
+final List parameterReferences =
configuration.getParameterReferences();
+for (final ParameterReference reference : parameterReferences) {
+final String referencedParamName =
reference.getParameterName();
+if (updatedParameters.containsKey(referencedParamName)) {
+propertyAffected = true;
+componentAffected = true;
+break;
+}
+}
+
+if (propertyAffected) {
+final String previousValue =
configuration.getEffectiveValue(previousParameterLookup);
+final String updatedValue =
configuration.getEffectiveValue(getParameterContext());
+
+// Check if the value of the property is truly affected. It's
possible that we could have a property configured as something like "#{a}#{b}"
+// Where parameter a = "abc-" and b = "cba". The update could
change a to "abc" and b to "-cba". As a result, the property value previously
was "abc-cba" and still is.
+// In such a case, we should not call onPropertyModified.
+final boolean propertyUpdated = !Objects.equals(previousValue,
updatedValue);
+if (propertyUpdated) {
+try {
+logger.debug("Parameter Context updated, resulting in
property {} of {} changing. Calling onPropertyModified().", propertyDescriptor,
this);
Review comment:
I believe we don't want that in case this is a sensitive parameter, right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] gerdansantos commented on issue #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Replication.
gerdansantos commented on issue #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Replication. URL: https://github.com/apache/nifi/pull/4065#issuecomment-597796989 @pvillard31 can you review this PR to ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7236) Fix MAP handling in JsonRowRecordReader
[ https://issues.apache.org/jira/browse/NIFI-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7236: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Fix MAP handling in JsonRowRecordReader > --- > > Key: NIFI-7236 > URL: https://issues.apache.org/jira/browse/NIFI-7236 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Fixing a bug in JsonRowRecordReader when reading a 'raw' record with a schema > that indicates that a field should be a Map. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 merged pull request #4122: NIFI-7236 - Fix MAP handling in JsonRowRecordReader
markap14 merged pull request #4122: NIFI-7236 - Fix MAP handling in JsonRowRecordReader URL: https://github.com/apache/nifi/pull/4122 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
[ https://issues.apache.org/jira/browse/NIFI-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7245: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > JMS processors should yield when connection factory initialisation failed > - > > Key: NIFI-7245 > URL: https://issues.apache.org/jira/browse/NIFI-7245 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > JMS processors yield properly when there is an exception during > getConnection() and the steps after (get session, send / receive messages). > They should also yield if the connection factory initialisation is > unsuccessful (eg. due to missing JMS client libraries or misconfiguration of > the factory class name). Otherwise the processors produce a lot of logs > continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
[ https://issues.apache.org/jira/browse/NIFI-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057300#comment-17057300 ] ASF subversion and git services commented on NIFI-7245: --- Commit 3de3ad40290ccd4c9e09d5c4fd03e83a6cbf0d86 in nifi's branch refs/heads/master from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3de3ad4 ] NIFI-7245: JMS processors yield when connection factory initialisation failed Signed-off-by: Pierre Villard This closes #4133. > JMS processors should yield when connection factory initialisation failed > - > > Key: NIFI-7245 > URL: https://issues.apache.org/jira/browse/NIFI-7245 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > JMS processors yield properly when there is an exception during > getConnection() and the steps after (get session, send / receive messages). > They should also yield if the connection factory initialisation is > unsuccessful (eg. due to missing JMS client libraries or misconfiguration of > the factory class name). Otherwise the processors produce a lot of logs > continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4133: NIFI-7245: JMS processors yield when connection factory initialisatio…
asfgit closed pull request #4133: NIFI-7245: JMS processors yield when connection factory initialisatio… URL: https://github.com/apache/nifi/pull/4133 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391145472
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/PropertyMatcher.java
##
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ProcessorNode;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import static
org.apache.nifi.web.search.attributematchers.AttributeMatcher.addIfMatching;
+
+public class PropertyMatcher implements AttributeMatcher {
+private static final String LABEL_NAME = "Property name";
+private static final String LABEL_VALUE = "Property value";
+private static final String LABEL_DESCRIPTION = "Property description";
+
+private final static String FILTER_NAME_PROPERTIES = "properties";
+private final static Set FILTER_VALUES_PROPERTIES_EXCLUSION = new
HashSet<>(Arrays.asList("no", "none", "false", "exclude", "0"));
+
+@Override
+public void match(final ProcessorNode component, final SearchQuery query,
final List matches) {
+final String searchTerm = query.getTerm();
+
+if (!propertiesAreFilteredOut(query)) {
+for (final Map.Entry entry :
component.getRawPropertyValues().entrySet()) {
+final PropertyDescriptor descriptor = entry.getKey();
+addIfMatching(searchTerm, descriptor.getName(), LABEL_NAME,
matches);
+addIfMatching(searchTerm, descriptor.getDescription(),
LABEL_DESCRIPTION, matches);
+
+// never include sensitive properties values in search results
+if (!descriptor.isSensitive()) {
+final String value = (entry.getValue() == null)
Review comment:
I don't think that this expression with the ternary operator is more
readable than the original.
If anything, I'd go with `Optional`, like this:
```java
final String value =
Optional.ofNullable(entry.getValue()).orElse(descriptor.getDefaultValue());
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391149208
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/SearchableMatcher.java
##
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.collections4.CollectionUtils;
+import org.apache.nifi.controller.FlowController;
+import org.apache.nifi.controller.ProcessorNode;
+import org.apache.nifi.nar.NarCloseable;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.registry.VariableRegistry;
+import org.apache.nifi.search.SearchContext;
+import org.apache.nifi.search.SearchResult;
+import org.apache.nifi.search.Searchable;
+import org.apache.nifi.web.controller.StandardSearchContext;
+import org.apache.nifi.web.search.query.SearchQuery;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Collection;
+import java.util.List;
+
+public class SearchableMatcher implements AttributeMatcher {
+private static final Logger LOGGER =
LoggerFactory.getLogger(SearchableMatcher.class);
+
+private FlowController flowController;
+private VariableRegistry variableRegistry;
+
+@Override
+public void match(final ProcessorNode component, final SearchQuery query,
final List matches) {
+final Processor processor = component.getProcessor();
+
+if (processor instanceof Searchable) {
+final Searchable searchable = (Searchable) processor;
+final String searchTerm = query.getTerm();
+final SearchContext context = new
StandardSearchContext(searchTerm, component,
flowController.getControllerServiceProvider(), variableRegistry);
+
+// search the processor using the appropriate thread context
classloader
+try (final NarCloseable narCloseable =
NarCloseable.withComponentNarLoader(flowController.getExtensionManager(),
component.getClass(), component.getIdentifier())) {
+final Collection searchResults =
searchable.search(context);
+
+if (CollectionUtils.isNotEmpty(searchResults)) {
+for (final SearchResult searchResult : searchResults) {
+matches.add(new
StringBuilder(searchResult.getLabel()).append(AttributeMatcher.SEPARATOR).append(searchResult.getMatch()).toString());
Review comment:
Unnecessary `StringBuilder`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391169992
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/BackPressureAttributeMatcher.java
##
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.connectable.Connection;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+public class BackPressureAttributeMatcher implements
AttributeMatcher {
+private static final String MATCH_PREFIX_SIZE = "Back pressure data size:
";
+private static final String MATCH_PREFIX_COUNT = "Back pressure count: ";
+private static final Set KEYWORDS = new HashSet<>(Arrays.asList( //
+"back pressure", //
+"pressure"));
+
+@Override
+public void match(final Connection component, final SearchQuery query,
final List matches) {
+if (containsKeyword(query)) {
+final String backPressureDataSize =
component.getFlowFileQueue().getBackPressureDataSizeThreshold();
+final Double backPressureBytes =
DataUnit.parseDataSize(backPressureDataSize, DataUnit.B);
+final long backPressureCount =
component.getFlowFileQueue().getBackPressureObjectThreshold();
+
+if (backPressureBytes > 0) {
+matches.add(MATCH_PREFIX_SIZE + backPressureDataSize);
+}
+
+if (backPressureCount > 0) {
+matches.add(MATCH_PREFIX_COUNT + backPressureCount);
+}
+}
+}
+
+private boolean containsKeyword(final SearchQuery query) {
+for (final String keyword : KEYWORDS) {
Review comment:
Minor suggestion: `stream().anyMatch(...)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391066175
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,162 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391141901
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/PropertyMatcher.java
##
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ProcessorNode;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import static
org.apache.nifi.web.search.attributematchers.AttributeMatcher.addIfMatching;
+
+public class PropertyMatcher implements AttributeMatcher {
+private static final String LABEL_NAME = "Property name";
+private static final String LABEL_VALUE = "Property value";
+private static final String LABEL_DESCRIPTION = "Property description";
+
+private final static String FILTER_NAME_PROPERTIES = "properties";
+private final static Set FILTER_VALUES_PROPERTIES_EXCLUSION = new
HashSet<>(Arrays.asList("no", "none", "false", "exclude", "0"));
+
+@Override
+public void match(final ProcessorNode component, final SearchQuery query,
final List matches) {
+final String searchTerm = query.getTerm();
+
+if (!propertiesAreFilteredOut(query)) {
+for (final Map.Entry entry :
component.getRawPropertyValues().entrySet()) {
+final PropertyDescriptor descriptor = entry.getKey();
+addIfMatching(searchTerm, descriptor.getName(), LABEL_NAME,
matches);
+addIfMatching(searchTerm, descriptor.getDescription(),
LABEL_DESCRIPTION, matches);
+
+// never include sensitive properties values in search results
+if (!descriptor.isSensitive()) {
+final String value = (entry.getValue() == null)
+? descriptor.getDefaultValue()
+: entry.getValue();
+
+// evaluate if the value matches the search criteria
+if (StringUtils.containsIgnoreCase(value, searchTerm)) {
+matches.add(new
StringBuilder(LABEL_VALUE).append(SEPARATOR).append(descriptor.getName()).append("
- ").append(value).toString());
Review comment:
Again, unnecessary `StringBuilder`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors URL: https://github.com/apache/nifi/pull/4123#discussion_r391150825 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/resources/nifi-web-api-context.xml ## @@ -75,6 +75,128 @@ + + + + + + + + + + + + + + + Review comment: Don't we need to inject `flowController` as well? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391121589
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/ComponentMatcher.java
##
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search;
+
+import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import javax.annotation.Nonnull;
+import java.util.Optional;
+
+public interface ComponentMatcher {
Review comment:
Indentation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors URL: https://github.com/apache/nifi/pull/4123#discussion_r391077893 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/resources/nifi-web-api-context.xml ## @@ -75,6 +75,128 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Review comment: `Labels` are not `Connectables`! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391167931
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/ExpirationAttributeMatcher.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.connectable.Connection;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+public class ExpirationAttributeMatcher implements
AttributeMatcher {
+private static final String MATCH_PREFIX = "FlowFile expiration: ";
+private static final Set KEYWORDS = new HashSet<>(Arrays.asList( //
+"expires", //
+"expiration"));
+
+@Override
+public void match(final Connection component, final SearchQuery query,
final List matches) {
+if (containsKeyword(query)) {
+final int expirationMillis =
component.getFlowFileQueue().getFlowFileExpiration(TimeUnit.MILLISECONDS);
+
+if (expirationMillis > 0) {
+matches.add(MATCH_PREFIX +
component.getFlowFileQueue().getFlowFileExpiration());
+}
+}
+}
+
+private boolean containsKeyword(final SearchQuery query) {
+for(final String keyword : KEYWORDS) {
Review comment:
Minor suggestion:
We already started using the stream API. How about
```java
return KEYWORDS.stream().anyMatch(keyword ->
StringUtils.containsIgnoreCase(keyword, query.getTerm()));
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
tpalfy commented on a change in pull request #4123: NIFI-7188: Adding filter
capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391108275
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
##
@@ -3106,8 +3106,8 @@ public ProvenanceEventDTO submitReplay(final Long
eventId) {
// -
@Override
-public SearchResultsDTO searchController(final String query) {
-return controllerFacade.search(query);
+public SearchResultsDTO searchController(final String query, final String
activeGroupId) {
Review comment:
Minor inconsistency: This parameter is named `currentGroupId` in the
interface.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] markap14 opened a new pull request #4134: NIFI-7242: When a Parameter is changed, any property referencing that…
markap14 opened a new pull request #4134: NIFI-7242: When a Parameter is
changed, any property referencing that…
URL: https://github.com/apache/nifi/pull/4134
… parameter should have its #onPropertyModified method called. Also renamed
Accumulo tests to integration tests because they start embedded servers and
connect to them, which caused failures in my environment. Also fixed a bug in
TestLengthDelimitedJournal because it was resulting in failures when building
locally as well.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (MINIFICPP-1139) Implement proper backpressure handling in ConsumeWindowEventLog
[ https://issues.apache.org/jira/browse/MINIFICPP-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Marmer resolved MINIFICPP-1139. Resolution: Fixed > Implement proper backpressure handling in ConsumeWindowEventLog > --- > > Key: MINIFICPP-1139 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1139 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Dániel Bakai >Assignee: Alex Marmer >Priority: Blocker > Fix For: 0.8.0 > > Time Spent: 10.5h > Remaining Estimate: 0h > > In backpressure situations it should unsubscribe from events, and after > bacpressure is relieved resubscribe using the bookmark/checkpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r391123308
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/AttributeMatcher.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.List;
+
+public interface AttributeMatcher {
+String SEPARATOR = ": ";
+
+void match(T component, SearchQuery query, List matches);
+
+static void addIfMatching(final String searchTerm, final String subject,
final String label, final List matches) {
+final String match = (label == null) //
Review comment:
Not in the happy path cases
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
[ https://issues.apache.org/jira/browse/NIFI-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-7245: -- Status: Patch Available (was: Open) > JMS processors should yield when connection factory initialisation failed > - > > Key: NIFI-7245 > URL: https://issues.apache.org/jira/browse/NIFI-7245 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > JMS processors yield properly when there is an exception during > getConnection() and the steps after (get session, send / receive messages). > They should also yield if the connection factory initialisation is > unsuccessful (eg. due to missing JMS client libraries or misconfiguration of > the factory class name). Otherwise the processors produce a lot of logs > continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
[ https://issues.apache.org/jira/browse/NIFI-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-7245: -- Component/s: Extensions > JMS processors should yield when connection factory initialisation failed > - > > Key: NIFI-7245 > URL: https://issues.apache.org/jira/browse/NIFI-7245 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > JMS processors yield properly when there is an exception during > getConnection() and the steps after (get session, send / receive messages). > They should also yield if the connection factory initialisation is > unsuccessful (eg. due to missing JMS client libraries or misconfiguration of > the factory class name). Otherwise the processors produce a lot of logs > continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] turcsanyip opened a new pull request #4133: NIFI-7245: JMS processors yield when connection factory initialisatio…
turcsanyip opened a new pull request #4133: NIFI-7245: JMS processors yield
when connection factory initialisatio…
URL: https://github.com/apache/nifi/pull/4133
…n failed
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (NIFI-7245) JMS processors should yield when connection factory initialisation failed
Peter Turcsanyi created NIFI-7245: - Summary: JMS processors should yield when connection factory initialisation failed Key: NIFI-7245 URL: https://issues.apache.org/jira/browse/NIFI-7245 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi JMS processors yield properly when there is an exception during getConnection() and the steps after (get session, send / receive messages). They should also yield if the connection factory initialisation is unsuccessful (eg. due to missing JMS client libraries or misconfiguration of the factory class name). Otherwise the processors produce a lot of logs continuously while running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm edited a comment on issue #731: WIP: MINIFICPP-1096 fix BackTrace, OOB indexing, tests, appveyor reporting
szaszm edited a comment on issue #731: WIP: MINIFICPP-1096 fix BackTrace, OOB indexing, tests, appveyor reporting URL: https://github.com/apache/nifi-minifi-cpp/pull/731#issuecomment-594662529 WIP: debugging C2VerifyHeartbeatAndStop on windows (flicker) edit: probably related to [MINIFICPP-1172](https://issues.apache.org/jira/browse/MINIFICPP-1172) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (MINIFICPP-1171) Fix "%ll" format strings in log lines
[
https://issues.apache.org/jira/browse/MINIFICPP-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057014#comment-17057014
]
Marton Szasz commented on MINIFICPP-1171:
-
I think this can be closed as the PR is merged
> Fix "%ll" format strings in log lines
> -
>
> Key: MINIFICPP-1171
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1171
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Malformed log strings don't print the value.
> As an example, this line of code:
> {code}./extensions/civetweb/processors/ListenHTTP.cpp:287:
> logger_->log_debug("ListenHTTP handling POST request of length %ll",
> req_info->content_length);{code}
> Results in the following log message:
> {code}[2020-03-03 17:00:29.710]
> [org::apache::nifi::minifi::processors::ListenHTTP::Handler] [debug]
> ListenHTTP handling POST request of length{code}
> (Compiled with g++8.3 on Debian Buster)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] ottobackwards commented on a change in pull request #4125: NIFI-7153 Adds ContentLengthFilter and DoSFilter
ottobackwards commented on a change in pull request #4125: NIFI-7153 Adds ContentLengthFilter and DoSFilter URL: https://github.com/apache/nifi/pull/4125#discussion_r390943550 ## File path: nifi-docs/src/main/asciidoc/administration-guide.adoc ## @@ -3247,6 +3247,9 @@ For example, when running in a Docker container or behind a proxy (e.g. localhos host[:port] that NiFi is bound to. |`nifi.web.proxy.context.path`|A comma separated list of allowed HTTP X-ProxyContextPath, X-Forwarded-Context, or X-Forwarded-Prefix header values to consider. By default, this value is blank meaning all requests containing a proxy context path are rejected. Configuring this property would allow requests where the proxy path is contained in this listing. +|`nifi.web.max.content.size`|The maximum size for regular PUT and POST requests. The default value is `10 MB`. Review comment: No problem, my confusion was around what "path" meant. I usually think of this as 'route' and associate path with like a download location. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #735: MINIFICPP-1158 - Event driven processors can starve each other
arpadboda closed pull request #735: MINIFICPP-1158 - Event driven processors can starve each other URL: https://github.com/apache/nifi-minifi-cpp/pull/735 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390908556
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,162 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390932356
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/query/RegexSearchQueryParser.java
##
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.query;
+
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.groups.ProcessGroup;
+
+import javax.annotation.Nonnull;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class RegexSearchQueryParser implements SearchQueryParser {
+private static final String REGEX =
"(?(([\\w]+\\:[\\w]+[\\s]+)*)(([\\w]+\\:[\\w]+){0,1}))(([\\w]+\\:[\\w]+)|(?.*))";
Review comment:
I take a look on this. I was experimenting a while until I found a pattern
works fine for everything but I can imagine there are simpler ones.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390927648
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/MatchEnriching.java
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search;
+
+import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
+import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
+
+import java.util.Optional;
+import java.util.function.Function;
+
+public class MatchEnriching implements Function {
Review comment:
I am not sure how about the customs in the project regarding (but I guess
you pointed out this because it is not the usual way), so I cannot really
oppose :) My primary intention with this is to provide an interface to able to
depend on it if necessary. So if it is a fitting approach, I would introduce an
interface which would "replace" `Function` here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390934339
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/attributematchers/BasicAttributeMatcher.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search.attributematchers;
+
+import org.apache.nifi.connectable.Connectable;
+import org.apache.nifi.web.search.query.SearchQuery;
+
+import java.util.List;
+
+import static
org.apache.nifi.web.search.attributematchers.AttributeMatcher.addIfMatching;
+
+public class BasicAttributeMatcher implements
AttributeMatcher {
Review comment:
I completely agree and I hesitated a lot. The reason I went this way is the
following: I wanted to reduce code duplication around search where it is
reasonable and originally I was hoping for one class with this behaviour. But
as the `getName` and `getIdentifier` methods in the different components are
not inherited from the same parent it ended with multiple classes with almost
the same behaviour. It looked a good idea to not add one more, but on second
though... It might also makes sense to follow an approach similar to the
`AttributeBasedComponentMatcher`. It comes with a somewhat more complex
instantiation but is more flexible. What is your opinion?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390903549
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,162 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390925256
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/search/ComponentMatcherFactory.java
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.search;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.connectable.Connectable;
+import org.apache.nifi.connectable.Connection;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.groups.RemoteProcessGroup;
+import org.apache.nifi.parameter.Parameter;
+import org.apache.nifi.parameter.ParameterContext;
+import org.apache.nifi.web.search.attributematchers.AttributeMatcher;
+
+import java.util.List;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+public class ComponentMatcherFactory {
+public ComponentMatcher getInstanceForConnectable(final
List> attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getIdentifier(), component -> component.getName());
+}
+
+public ComponentMatcher getInstanceForConnection(final
List> attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getIdentifier(), new GetConnectionName());
+}
+
+public ComponentMatcher getInstanceForParameter(final
List> attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getDescriptor().getName(), component ->
component.getDescriptor().getName());
+}
+
+public ComponentMatcher
getInstanceForParameterContext(final List>
attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getIdentifier(), component -> component.getName());
+}
+
+public ComponentMatcher getInstanceForProcessGroup(final
List> attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getIdentifier(), component -> component.getName());
+}
+
+public ComponentMatcher
getInstanceForRemoteProcessGroup(final
List> attributeMatchers) {
+return new AttributeBasedComponentMatcher<>(attributeMatchers,
component -> component.getIdentifier(), component -> component.getName());
+}
+
+private static class GetConnectionName implements Function {
+private static final String DEFAULT_NAME_PREFIX = "From source ";
+private static final String SEPARATOR = ", ";
+
+public String apply(final Connection component) {
+String result = null;
+
+if (StringUtils.isNotBlank(component.getName())) {
+result = component.getName();
+} else if (!component.getRelationships().isEmpty()) {
+result = component.getRelationships().stream() //
Review comment:
Valid point. However it looks somewhat harder to read
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390923477
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,150 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390910512
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,162 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390915238
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,162 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors URL: https://github.com/apache/nifi/pull/4123#discussion_r390859318 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java ## @@ -147,8 +148,8 @@ // properties private NiFiProperties properties; private DtoFactory dtoFactory; -private VariableRegistry variableRegistry; private ControllerSearchService controllerSearchService; +private SearchQueryParser searchQueryParser; Review comment: It is a user input parser, and based on that I think it is belongs more to the controller side than the service side. I think it is more clear from the perspective of boundaries if the service is not responsible to parse the user input string but working with a request object. Also from the practical side: as the "search" and the "searchParameters" are called separately from the controller facade and they both need the user input, it looks unnecessary to parse it twice within the search methods. (This separation that they are distinct calls is a decision from the original code which I did not intend to change) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] simonbence commented on a change in pull request #4123: NIFI-7188: Adding filter capabilities into search & prerequisite refactors
simonbence commented on a change in pull request #4123: NIFI-7188: Adding
filter capabilities into search & prerequisite refactors
URL: https://github.com/apache/nifi/pull/4123#discussion_r390921644
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java
##
@@ -16,564 +16,150 @@
*/
package org.apache.nifi.web.controller;
-import org.apache.commons.collections4.CollectionUtils;
-import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.authorization.Authorizer;
import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.authorization.user.NiFiUser;
-import org.apache.nifi.authorization.user.NiFiUserUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.validation.ValidationStatus;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
-import org.apache.nifi.controller.ScheduledState;
import org.apache.nifi.controller.label.Label;
-import org.apache.nifi.controller.queue.FlowFileQueue;
-import org.apache.nifi.flowfile.FlowFilePrioritizer;
import org.apache.nifi.groups.ProcessGroup;
import org.apache.nifi.groups.RemoteProcessGroup;
-import org.apache.nifi.nar.NarCloseable;
import org.apache.nifi.parameter.Parameter;
import org.apache.nifi.parameter.ParameterContext;
-import org.apache.nifi.parameter.ParameterContextManager;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.Processor;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.registry.ComponentVariableRegistry;
-import org.apache.nifi.registry.VariableDescriptor;
-import org.apache.nifi.registry.VariableRegistry;
-import org.apache.nifi.remote.PublicPort;
-import org.apache.nifi.scheduling.ExecutionNode;
-import org.apache.nifi.scheduling.SchedulingStrategy;
-import org.apache.nifi.search.SearchContext;
-import org.apache.nifi.search.SearchResult;
-import org.apache.nifi.search.Searchable;
import org.apache.nifi.web.api.dto.search.ComponentSearchResultDTO;
import org.apache.nifi.web.api.dto.search.SearchResultGroupDTO;
import org.apache.nifi.web.api.dto.search.SearchResultsDTO;
+import org.apache.nifi.web.search.ComponentMatcher;
+import org.apache.nifi.web.search.MatchEnriching;
+import org.apache.nifi.web.search.query.SearchQuery;
-import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
-import java.util.Map;
+import java.util.Optional;
import java.util.Set;
-import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
/**
* NiFi web controller's helper service that implements component search.
*/
public class ControllerSearchService {
+private final static String FILTER_NAME_GROUP = "group";
+private final static String FILTER_NAME_SCOPE = "scope";
+private final static String FILTER_SCOPE_VALUE_HERE = "here";
+
private FlowController flowController;
private Authorizer authorizer;
-private VariableRegistry variableRegistry;
-
-/**
- * Searches term in the controller beginning from a given process group.
- *
- * @param results Search results
- * @param search The search term
- * @param group The init process group
- */
-public void search(final SearchResultsDTO results, final String search,
final ProcessGroup group) {
-final NiFiUser user = NiFiUserUtils.getNiFiUser();
-
-if (group.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO groupMatch = search(search, group);
-if (groupMatch != null) {
-// get the parent group, not the current one
-groupMatch.setParentGroup(buildResultGroup(group.getParent(),
user));
-
groupMatch.setVersionedGroup(buildVersionedGroup(group.getParent(), user));
-results.getProcessGroupResults().add(groupMatch);
-}
-}
-
-for (final ProcessorNode procNode : group.getProcessors()) {
-if (procNode.isAuthorized(authorizer, RequestAction.READ, user)) {
-final ComponentSearchResultDTO match = search(search,
procNode);
-if (match != null) {
-match.setGroupId(group.getIdentifier());
-match.setParentGroup(buildResultGroup(group, user));
-match.setVersionedGroup(buildVersionedGroup(group, user));
-results.getProcessorResults().add(match);
-}
-}
-}
-
-for (final Connection connection :
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #745: MINIFICPP-1164 patch libarchive-3.3.2 to avoid Werror
arpadboda closed pull request #745: MINIFICPP-1164 patch libarchive-3.3.2 to avoid Werror URL: https://github.com/apache/nifi-minifi-cpp/pull/745 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] pvillard31 commented on a change in pull request #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
pvillard31 commented on a change in pull request #4131: NIFI-7176 added
InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
URL: https://github.com/apache/nifi/pull/4131#discussion_r390889303
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
##
@@ -182,6 +182,7 @@
.name("Connection Timeout")
.description("Max wait time for connection to remote service.")
.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
```suggestion
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] pvillard31 commented on a change in pull request #4131: NIFI-7176 added InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
pvillard31 commented on a change in pull request #4131: NIFI-7176 added
InvokeHTTP support parametric READ.TIMEOUT and CONNEC…
URL: https://github.com/apache/nifi/pull/4131#discussion_r390889395
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
##
@@ -190,6 +191,7 @@
.name("Read Timeout")
.description("Max wait time for response from remote service.")
.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
```suggestion
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-7050) ConsumeJMS is not yielded in case of exception
[
https://issues.apache.org/jira/browse/NIFI-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7050:
-
Fix Version/s: 1.11.4
1.12.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConsumeJMS is not yielded in case of exception
> --
>
> Key: NIFI-7050
> URL: https://issues.apache.org/jira/browse/NIFI-7050
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.10.0
>Reporter: Gardella Juan Pablo
>Assignee: Gardella Juan Pablo
>Priority: Minor
> Fix For: 1.12.0, 1.11.4
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> If any exception happens when ConsumerJMS tries to read messages, the process
> tries again immediately.
> {code:java}
> try {
> consumer.consume(destinationName, errorQueueName, durable,
> shared, subscriptionName, charset, new ConsumerCallback() {
> @Override
> public void accept(final JMSResponse response) {
> if (response == null) {
> return;
> }
> FlowFile flowFile = processSession.create();
> flowFile = processSession.write(flowFile, out ->
> out.write(response.getMessageBody()));
> final Map jmsHeaders =
> response.getMessageHeaders();
> final Map jmsProperties =
> response.getMessageProperties();
> flowFile =
> ConsumeJMS.this.updateFlowFileAttributesWithJMSAttributes(jmsHeaders,
> flowFile, processSession);
> flowFile =
> ConsumeJMS.this.updateFlowFileAttributesWithJMSAttributes(jmsProperties,
> flowFile, processSession);
> flowFile = processSession.putAttribute(flowFile,
> JMS_SOURCE_DESTINATION_NAME, destinationName);
> processSession.getProvenanceReporter().receive(flowFile,
> destinationName);
> processSession.putAttribute(flowFile, JMS_MESSAGETYPE,
> response.getMessageType());
> processSession.transfer(flowFile, REL_SUCCESS);
> processSession.commit();
> }
> });
> } catch(Exception e) {
> consumer.setValid(false);
> throw e; // for backward compatibility with exception handling in
> flows
> }
> }
> {code}
> It should call {{context.yield}} in exception block. Notice
> [PublishJMS|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java#L166]
> is yielded in the same scenario. It is requires to do in the ConsumeJMS
> processor only.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7243) SFTP Processors - Exhausted available authentication methods
[ https://issues.apache.org/jira/browse/NIFI-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7243: - Component/s: (was: Core Framework) Extensions > SFTP Processors - Exhausted available authentication methods > > > Key: NIFI-7243 > URL: https://issues.apache.org/jira/browse/NIFI-7243 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.10.0, 1.11.0, 1.11.1, 1.11.3 > Environment: PRO >Reporter: Raúl >Priority: Critical > Labels: SFTP, listsftp > > In version 1.8 this works correctly. In superiors no. > > Any of the processors: ListSFTP, FetchSFTP, GetSFTP, PutSFTP when connecting > to an sftp shows the error: > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > at net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) > > These tests that I commented have been made from nifi, both in windows and > linux environment and fails in both. > 1 - Strict Host Key Checking: false > 2 - Strict Host Key Checking: true > Host Key File: C:\Users\usuario\.ssh\known_hosts --> (ssh-rsa, > ssh-dss and ecdsa-sha2-nistp256) > > From the machines, both in windows and in linux, if an sftp is made from the > command line it connects without problems. But with Nifi he fails to connect. > > Any idea about how can I resolve this error? > > Thank you in advance, > log > - > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods2020-03-10 15:12:51,596 ERROR [Timer-Driven Process > Thread-3] o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methodsnet.schmizz.sshj.userauth.UserAuthException: Exhausted > available authentication methods at > net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) at > org.apache.nifi.processors.standard.util.SFTPTransfer.getSFTPClient(SFTPTransfer.java:606) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:233) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:196) > at > org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) > at > org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:146) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)2020-03-10 15:12:51,601 DEBUG > [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] has chosen to yield its > resources; will not be scheduled to run again for 1000
[jira] [Commented] (NIFI-7050) ConsumeJMS is not yielded in case of exception
[
https://issues.apache.org/jira/browse/NIFI-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056811#comment-17056811
]
ASF subversion and git services commented on NIFI-7050:
---
Commit 89d8b877f9adc3f474983c4dec45bf4586a6baef in nifi's branch
refs/heads/master from Gardella Juan Pablo
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=89d8b87 ]
NIFI-7050 ConsumeJMS is not yielded in case of exception
This closes #4004.
Signed-off-by: Peter Turcsanyi
> ConsumeJMS is not yielded in case of exception
> --
>
> Key: NIFI-7050
> URL: https://issues.apache.org/jira/browse/NIFI-7050
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.10.0
>Reporter: Gardella Juan Pablo
>Assignee: Gardella Juan Pablo
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If any exception happens when ConsumerJMS tries to read messages, the process
> tries again immediately.
> {code:java}
> try {
> consumer.consume(destinationName, errorQueueName, durable,
> shared, subscriptionName, charset, new ConsumerCallback() {
> @Override
> public void accept(final JMSResponse response) {
> if (response == null) {
> return;
> }
> FlowFile flowFile = processSession.create();
> flowFile = processSession.write(flowFile, out ->
> out.write(response.getMessageBody()));
> final Map jmsHeaders =
> response.getMessageHeaders();
> final Map jmsProperties =
> response.getMessageProperties();
> flowFile =
> ConsumeJMS.this.updateFlowFileAttributesWithJMSAttributes(jmsHeaders,
> flowFile, processSession);
> flowFile =
> ConsumeJMS.this.updateFlowFileAttributesWithJMSAttributes(jmsProperties,
> flowFile, processSession);
> flowFile = processSession.putAttribute(flowFile,
> JMS_SOURCE_DESTINATION_NAME, destinationName);
> processSession.getProvenanceReporter().receive(flowFile,
> destinationName);
> processSession.putAttribute(flowFile, JMS_MESSAGETYPE,
> response.getMessageType());
> processSession.transfer(flowFile, REL_SUCCESS);
> processSession.commit();
> }
> });
> } catch(Exception e) {
> consumer.setValid(false);
> throw e; // for backward compatibility with exception handling in
> flows
> }
> }
> {code}
> It should call {{context.yield}} in exception block. Notice
> [PublishJMS|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java#L166]
> is yielded in the same scenario. It is requires to do in the ConsumeJMS
> processor only.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4004: NIFI-7050 ConsumeJMS is not yielded in case of exception
asfgit closed pull request #4004: NIFI-7050 ConsumeJMS is not yielded in case of exception URL: https://github.com/apache/nifi/pull/4004 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on issue #4004: NIFI-7050 ConsumeJMS is not yielded in case of exception
turcsanyip commented on issue #4004: NIFI-7050 ConsumeJMS is not yielded in case of exception URL: https://github.com/apache/nifi/pull/4004#issuecomment-597542285 @gardellajuanpablo Thanks for the contribution and fixing this issue. The code change and the test looks good to me. Also tested with a simple flow. Merging to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-7239) Upgrade the Hive 3 bundle to use Apache Hive 3.1.2
[ https://issues.apache.org/jira/browse/NIFI-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056782#comment-17056782 ] ASF subversion and git services commented on NIFI-7239: --- Commit 9cb85a4fd0b42d0b78e4c9a957cd6f659e1c91d0 in nifi's branch refs/heads/master from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9cb85a4 ] NIFI-7239: Upgrade the Hive 3 bundle to use Apache Hive 3.1.2 Signed-off-by: Pierre Villard This closes #4129. > Upgrade the Hive 3 bundle to use Apache Hive 3.1.2 > -- > > Key: NIFI-7239 > URL: https://issues.apache.org/jira/browse/NIFI-7239 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NiFi's Hive 3 processors use 3.1.0 Hive client currently. > Hive 3.1.1 and 3.1.2 contain a lot of fixes. > Eg. this critical one: HIVE-20979 - Fix memory leak in hive streaming. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7239) Upgrade the Hive 3 bundle to use Apache Hive 3.1.2
[ https://issues.apache.org/jira/browse/NIFI-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7239: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade the Hive 3 bundle to use Apache Hive 3.1.2 > -- > > Key: NIFI-7239 > URL: https://issues.apache.org/jira/browse/NIFI-7239 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > NiFi's Hive 3 processors use 3.1.0 Hive client currently. > Hive 3.1.1 and 3.1.2 contain a lot of fixes. > Eg. this critical one: HIVE-20979 - Fix memory leak in hive streaming. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4129: NIFI-7239: Upgrade the Hive 3 bundle to use Apache Hive 3.1.2
asfgit closed pull request #4129: NIFI-7239: Upgrade the Hive 3 bundle to use Apache Hive 3.1.2 URL: https://github.com/apache/nifi/pull/4129 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-4970) EOF Exception in InvokeHttp when body's response is empty
[
https://issues.apache.org/jira/browse/NIFI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-4970:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> EOF Exception in InvokeHttp when body's response is empty
> --
>
> Key: NIFI-4970
> URL: https://issues.apache.org/jira/browse/NIFI-4970
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.5.0, 1.11.3
> Environment: Apache NiFi - Version 1.5.0.3.1.1.0-35
>Reporter: Francois Brillon
>Assignee: Eduardo Mota Fontes
>Priority: Major
> Fix For: 1.12.0, 1.11.4
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> *Description*
> A POST to an API that returns an empty body on success (status code 200) will
> generate an EOF Exception, causing the processor to always remain in error
> and routing all flow files to failures, even if the API call succeeded.
> An example of such API is the Streaming API of PowerBI:
> [https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming]
> *Exception Stack Traces*
> When the +property "Put Response Body In Attribute" is not set+, the
> exception is as follows:
> {code:java}
> rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due
> to java.io.EOFException: org.apache.nifi.pro
> cessor.exception.FlowFileAccessException: Failed to import data from
> buffer(okio.GzipSource@159311b9).inputStream() for St
> andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p
> b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io
> .EOFException
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import
> data from buffer(okio.GzipSource@159311b9).i
> nputStream() for
> StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio
> n-20180313-182039.pb,size=0] due to
> org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create
> ContentC
> laim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935)
> ... 13 common frames omitted
> Caused by: java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at okio.GzipSource.consumeHeader(GzipSource.java:114)
> at okio.GzipSource.read(GzipSource.java:73)
> at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932)
> ... 13 common frames omitted
> {code}
> When +"Put Body Response in Attribute" property is set+, the exception is as
> follows :
> {code:java}
> 2018-03-13 18:32:13,165 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=5aae3a34-35d9-1268-6e6d-b774d4467e5a] Routing to Failure due to
> exception: java.io.EOFException: java.io.EOFException
>
[jira] [Commented] (NIFI-4970) EOF Exception in InvokeHttp when body's response is empty
[
https://issues.apache.org/jira/browse/NIFI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056771#comment-17056771
]
ASF subversion and git services commented on NIFI-4970:
---
Commit a679e88b6f4f0e5f800aba5a6891494bdbdfb193 in nifi's branch
refs/heads/master from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a679e88 ]
NIFI-4970 - Add a property to deal with empty GZIP HTTP response
Signed-off-by: Pierre Villard
This closes #4127.
> EOF Exception in InvokeHttp when body's response is empty
> --
>
> Key: NIFI-4970
> URL: https://issues.apache.org/jira/browse/NIFI-4970
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.5.0, 1.11.3
> Environment: Apache NiFi - Version 1.5.0.3.1.1.0-35
>Reporter: Francois Brillon
>Assignee: Eduardo Mota Fontes
>Priority: Major
> Fix For: 1.12.0, 1.11.4
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> *Description*
> A POST to an API that returns an empty body on success (status code 200) will
> generate an EOF Exception, causing the processor to always remain in error
> and routing all flow files to failures, even if the API call succeeded.
> An example of such API is the Streaming API of PowerBI:
> [https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming]
> *Exception Stack Traces*
> When the +property "Put Response Body In Attribute" is not set+, the
> exception is as follows:
> {code:java}
> rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due
> to java.io.EOFException: org.apache.nifi.pro
> cessor.exception.FlowFileAccessException: Failed to import data from
> buffer(okio.GzipSource@159311b9).inputStream() for St
> andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p
> b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io
> .EOFException
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import
> data from buffer(okio.GzipSource@159311b9).i
> nputStream() for
> StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio
> n-20180313-182039.pb,size=0] due to
> org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create
> ContentC
> laim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935)
> ... 13 common frames omitted
> Caused by: java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at okio.GzipSource.consumeHeader(GzipSource.java:114)
> at okio.GzipSource.read(GzipSource.java:73)
> at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932)
> ... 13 common frames omitted
> {code}
> When +"Put Body Response in Attribute" property is set+, the
[GitHub] [nifi] asfgit closed pull request #4127: NIFI-4970 - Add a property to deal with empty GZIP HTTP response
asfgit closed pull request #4127: NIFI-4970 - Add a property to deal with empty GZIP HTTP response URL: https://github.com/apache/nifi/pull/4127 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
