[jira] [Updated] (NIFI-12773) Add 'join' and 'anchored' RecordPath functions
[
https://issues.apache.org/jira/browse/NIFI-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-12773:
--
Status: Patch Available (was: Open)
> Add 'join' and 'anchored' RecordPath functions
> --
>
> Key: NIFI-12773
> URL: https://issues.apache.org/jira/browse/NIFI-12773
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I've come across two functions that would make flow design much simpler in
> RecordPath.
> The first one, 'join' would be similar to the 'concat' method but provides a
> delimiter between each element instead of just smashing the values together.
> The other provides the ability to anchor the context node while evaluating a
> RecordPath. For example, given the following record:
> {code:java}
> {
> "id": "1234",
> "elements": [{
> "name": "book",
> "color": "red"
> }, {
> "name": "computer",
> "color": "black"
> }]
> } {code}
> We should be able to use:
> {code:java}
> anchored(/elements, concat(/name, ': ', /color)) {code}
> In order to obtain an array of 2 elements:
> {code:java}
> book: red {code}
> and
> {code:java}
> computer: black {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12773: Added join and anchored RecordPath function [nifi]
markap14 opened a new pull request, #8391: URL: https://github.com/apache/nifi/pull/8391 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12773) Add 'join' and 'anchored' RecordPath functions
Mark Payne created NIFI-12773:
-
Summary: Add 'join' and 'anchored' RecordPath functions
Key: NIFI-12773
URL: https://issues.apache.org/jira/browse/NIFI-12773
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 2.0.0
I've come across two functions that would make flow design much simpler in
RecordPath.
The first one, 'join' would be similar to the 'concat' method but provides a
delimiter between each element instead of just smashing the values together.
The other provides the ability to anchor the context node while evaluating a
RecordPath. For example, given the following record:
{code:java}
{
"id": "1234",
"elements": [{
"name": "book",
"color": "red"
}, {
"name": "computer",
"color": "black"
}]
} {code}
We should be able to use:
{code:java}
anchored(/elements, concat(/name, ': ', /color)) {code}
In order to obtain an array of 2 elements:
{code:java}
book: red {code}
and
{code:java}
computer: black {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816246#comment-17816246 ] Tom Brisland commented on NIFI-12772: - I'll do some manual testing tomorrow but I did think from an initial browse that it was already implemented > Remote poll batch size not exposed for ListSFTP > --- > > Key: NIFI-12772 > URL: https://issues.apache.org/jira/browse/NIFI-12772 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Tom Brisland >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > I was planning on adding support for a batch size in ListSFTP due to some > issues we're seeing with a large number of files on an SFTP server. > Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all > the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12772 Expose REMOTE_POLL_BATCH_SIZE property for ListSFTP [nifi]
tombrisland commented on PR #8390: URL: https://github.com/apache/nifi/pull/8390#issuecomment-1936728485 Need to do some manual testing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12772 Expose REMOTE_POLL_BATCH_SIZE property for ListSFTP [nifi]
tombrisland opened a new pull request, #8390: URL: https://github.com/apache/nifi/pull/8390 # Summary [NIFI-12772](https://issues.apache.org/jira/browse/NIFI-12772) Allows use of the REMOTE_POLL_BATCH_SIZE property in the ListSFTP processor. Previously this was supported but not exposed for configuration. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816245#comment-17816245 ] Joe Witt commented on NIFI-12772: - The challenge here is whether the underlying library makes it possible to control that. If it does then great. If not then...not sure how to solve. A similar difficulty existed back in the day for ListFile or equivalent cases. Java itself didnt expose a way to control this. You could bring down a system just by making a listing call in a large directory. New IO mechanisms resolved that scenario but SFTP is a different animal. Love the idea just need to make sure the library exposes that power. > Remote poll batch size not exposed for ListSFTP > --- > > Key: NIFI-12772 > URL: https://issues.apache.org/jira/browse/NIFI-12772 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Tom Brisland >Priority: Minor > > I was planning on adding support for a batch size in ListSFTP due to some > issues we're seeing with a large number of files on an SFTP server. > Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all > the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
Tom Brisland created NIFI-12772: --- Summary: Remote poll batch size not exposed for ListSFTP Key: NIFI-12772 URL: https://issues.apache.org/jira/browse/NIFI-12772 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.25.0 Reporter: Tom Brisland I was planning on adding support for a batch size in ListSFTP due to some issues we're seeing with a large number of files on an SFTP server. Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-6344) Add Failure Relationship to UpdateAttribute
[ https://issues.apache.org/jira/browse/NIFI-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816244#comment-17816244 ] Joe Witt commented on NIFI-6344: Given the new capabilities for migrating configs in NiFi 2.0 we can fix this. Add a property to UpdateAttribute that is 'Failure Strategy' and the options are 'rollback' or 'route to failure'. If that property is set with rollback it behaves like it does now and I recommend that remain the default. If that property is set to 'route to failure' then we add a relationship which needs to be set which is of course called 'failure'. For flows being migrated from a version before this behavior was available to a version that has this capability we just set the value of this parameter to our default. This lets existing flows migrate over just fine. It lets us give users a failure path for the cases they want one. It lets us keep the vast majority of flows and uses of this where failure is not relevant stay clean. And it handles migration. The processor needs to be updated to catch the exceptions and then follow this logic. Today it just lets it fly to the framework which causes the processor to yield and penalizes the flowfile for the default time. When now catching the problem we should just avoid yielding and instead penalize the specific offending flowfile which lets everything else operate super fast. Thanks to Mark Payne for the chat on this. > Add Failure Relationship to UpdateAttribute > --- > > Key: NIFI-6344 > URL: https://issues.apache.org/jira/browse/NIFI-6344 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.9.2 >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Labels: attribute, backwards-compatibility, expression-language, > routing > Time Spent: 3h 10m > Remaining Estimate: 0h > > EL makes it possible for an UpdateAttribute processor to fail. When this > happens the FlowFile is rolled back, and there is no way to route it to > handle the failure automatically. > Considerations: > UpdateAttribute is used in probably all but the simplest of flows, thus any > change made to support a failure relationship must be handled delicately. The > goal of this change is for users to have no change in functionality unless > they specifically configure it. > Proposal: > It was proposed on the Slack channel to create the failure relationship, but > default it to auto-terminate. This is a good start, but without further work > would result in a change in functionality. I propose that we will default to > auto-terminate, but also detect this behavior in the code. If the Failure > relationship is set to auto-terminate then we will rollback the transaction. > The only downside I see with this is you can't actually auto-terminate > Failures without the addition of another property, such as Failure Behavior: > Route to Failure and Rollback options. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5448) Failed EL date parsing live-locks processors without a failure relationship
[
https://issues.apache.org/jira/browse/NIFI-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-5448.
Resolution: Duplicate
Will address the fundamental concern in this other jira
> Failed EL date parsing live-locks processors without a failure relationship
> ---
>
> Key: NIFI-5448
> URL: https://issues.apache.org/jira/browse/NIFI-5448
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: David Koster
>Assignee: Mike Thomsen
>Priority: Major
>
> Processors that utilize the Expression Language need to always present a
> failure relationship.
> If a processor with only a success relationship, for example UpdateAttribute,
> utilizes the expression language to perform type coercion to a date and
> fails, the processor will be unable to dispose of the FlowFile and remain
> blocked indefinitely.
> Recreation flow:
> GenerateFlowFile -> Update Attribute #1 -> Update Attribute #2 -> Anything
> Update Attribute #1 - test = "Hello World"
> Update Attribute #2 - test = ${test:toDate('-MM-dd')}
>
> Generates an IllegalAttributeException on UpdateAttribute.
>
> The behavior should match numerical type coercion and silently skip the
> processing or offer failure relationships on processors supporting EL
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12765 Remove Apache Ranger modules [nifi]
exceptionfactory opened a new pull request, #8389: URL: https://github.com/apache/nifi/pull/8389 # Summary [NIFI-12765](https://issues.apache.org/jira/browse/NIFI-12765) Removes Apache Ranger modules from NiFi and NiFi Registry based on incompatibilities with Jetty 12 and related libraries. Ranger libraries depend on Jetty 9, which was marked as End of Community Support in 2022. Although Jetty 10 and 11 provided some level of compatibility with earlier versions, both of those versions of End of Community Support as of January 2024. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12771) Restore Client-side KMS encryption strategy in StandardS3EncryptionService
[ https://issues.apache.org/jira/browse/NIFI-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12771: --- Status: Patch Available (was: Open) > Restore Client-side KMS encryption strategy in StandardS3EncryptionService > -- > > Key: NIFI-12771 > URL: https://issues.apache.org/jira/browse/NIFI-12771 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-12144 partially removed "Client-side KMS" encryption strategy. The > strategy class and some code were deleted but the option is still available > on the UI, just not works. > It seems it happened unintentionally. Bring back the functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12771 Restore Client-side KMS encryption strategy in StandardS3E… [nifi]
turcsanyip opened a new pull request, #8388: URL: https://github.com/apache/nifi/pull/8388 …ncryptionService # Summary [NIFI-12771](https://issues.apache.org/jira/browse/NIFI-12771) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12770) Deprecate Apache Ranger Integration for Removal
[ https://issues.apache.org/jira/browse/NIFI-12770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12770: Status: Patch Available (was: Open) > Deprecate Apache Ranger Integration for Removal > --- > > Key: NIFI-12770 > URL: https://issues.apache.org/jira/browse/NIFI-12770 > Project: Apache NiFi > Issue Type: Task >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Apache Ranger integration should be deprecated on the support branch for > subsequent removal on the main branch due to incompatibilities with Jetty 12 > and related libraries. The Ranger plugins require Jetty 9, which was marked > as [End of Community > Support|https://github.com/jetty/jetty.project/issues/7958] in June 2022. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12771) Restore Client-side KMS encryption strategy in StandardS3EncryptionService
Peter Turcsanyi created NIFI-12771: -- Summary: Restore Client-side KMS encryption strategy in StandardS3EncryptionService Key: NIFI-12771 URL: https://issues.apache.org/jira/browse/NIFI-12771 Project: Apache NiFi Issue Type: Bug Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi NIFI-12144 partially removed "Client-side KMS" encryption strategy. The strategy class and some code were deleted but the option is still available on the UI, just not works. It seems it happened unintentionally. Bring back the functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12770 Deprecate Ranger Authorizers for Removal [nifi]
exceptionfactory opened a new pull request, #8387: URL: https://github.com/apache/nifi/pull/8387 # Summary [NIFI-12770](https://issues.apache.org/jira/browse/NIFI-12770) Deprecates Apache Ranger integration on the support branch for subsequent removal on the main branch, due to incompatibility with Jetty 12 and related libraries. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12770) Deprecate Apache Ranger Integration for Removal
David Handermann created NIFI-12770: --- Summary: Deprecate Apache Ranger Integration for Removal Key: NIFI-12770 URL: https://issues.apache.org/jira/browse/NIFI-12770 Project: Apache NiFi Issue Type: Task Reporter: David Handermann Assignee: David Handermann Apache Ranger integration should be deprecated on the support branch for subsequent removal on the main branch due to incompatibilities with Jetty 12 and related libraries. The Ranger plugins require Jetty 9, which was marked as [End of Community Support|https://github.com/jetty/jetty.project/issues/7958] in June 2022. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12767) Provenance - General error handling in API responses
[ https://issues.apache.org/jira/browse/NIFI-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12767: --- Status: Patch Available (was: Open) > Provenance - General error handling in API responses > > > Key: NIFI-12767 > URL: https://issues.apache.org/jira/browse/NIFI-12767 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Introduce general error handling (banner, full screen, snackbar) for APIs > calls in the Provenance page. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman opened a new pull request, #8386: URL: https://github.com/apache/nifi/pull/8386 NIFI-12767: - Error handling in Provenance and Lineage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12765) Nifi and nifi registry ranger audit is broken
[
https://issues.apache.org/jira/browse/NIFI-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816227#comment-17816227
]
Joe Witt commented on NIFI-12765:
-
This JIRA seems strongly related to NIFI-12738 and should probably be combined.
The comments on that JIRA apply here as well
https://issues.apache.org/jira/browse/NIFI-12738
Based on my recent reading it appears Ranger depends on Jetty 9 so it is
actually surprising Jetty 10 or 11 works for it in any capacity but notably we
clearly seem to have no such tests/validations for this and it is not being
maintained in our apache nifi codebase. Also notably Jetty 9, 10, 11 are all
end of life from a community support point of view. My recommendations on the
other JIRA apply here as well.
> Nifi and nifi registry ranger audit is broken
> -
>
> Key: NIFI-12765
> URL: https://issues.apache.org/jira/browse/NIFI-12765
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2, 2.0.0
>Reporter: Zoltán Kornél Török
>Assignee: Zoltán Kornél Török
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Bug description
> Currently ranger plugins are not reporting audit events into ranger.
> h2. Investigation
> In the nifi log I found the following ("classic") NoClassDefFoundError:
> {code:java}
> ERROR org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect
> to Solr server.
> ZooKeepers=cfm-oudjal-dd-master0.cfm-5pax.svbr-nqvp.int.cldr.work:2181/solr-infrajava.lang.NoClassDefFoundError:
> org/eclipse/jetty/client/util/SPNEGOAuthentication
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:168)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:227)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: java.lang.ClassNotFoundException:
> org.eclipse.jetty.client.util.SPNEGOAuthentication
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 5 common frames omitted {code}
> As you can see ranger-audit depends on solr client which depends on jetty
> client.
> The problem is that solr client class use
> org.eclipse.jetty.client.util.SPNEGOAuthentication -
> [https://github.infra.cloudera.com/CDH/solr/blob/solr9-master/solr/solrj/src/java/org/apache/solr/client/solrj/impl/Krb5HttpClientBuilder.java#L46]
> However in case jetty 12.x line, this class is moved to another package:
> [https://github.com/jetty/jetty.project/commit/a1c5cefd0d5657df04e5364cca9315aa4e2a1aef]
>
> So the problem exist, since jetty version upgraded to 12
> h2. Proposed solution
> Sadly there is no available solr client (or ranger client), which haven't had
> this dependency. The only solution what I found (and propose in my pr) is to
> override jetty version in case of ranger plugins to jetty line 11, where this
> class is not moved. I tested it on my environment and the audit logging to
> ranger worked well with that version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12769) Update Copyright for 2024
[ https://issues.apache.org/jira/browse/NIFI-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12769: Fix Version/s: 2.0.0 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update Copyright for 2024 > - > > Key: NIFI-12769 > URL: https://issues.apache.org/jira/browse/NIFI-12769 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0, 1.26.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12769) Update Copyright for 2024
[ https://issues.apache.org/jira/browse/NIFI-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816225#comment-17816225 ] ASF subversion and git services commented on NIFI-12769: Commit 3baf052aae18c97c7f2d39a4fc681e5cc8406139 in nifi's branch refs/heads/support/nifi-1.x from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3baf052aae ] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers This closes #8385. Signed-off-by: Joseph Witt > Update Copyright for 2024 > - > > Key: NIFI-12769 > URL: https://issues.apache.org/jira/browse/NIFI-12769 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers [nifi]
joewitt commented on PR #8384: URL: https://github.com/apache/nifi/pull/8384#issuecomment-1936624321 merged - thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers (1.x) [nifi]
joewitt commented on PR #8385: URL: https://github.com/apache/nifi/pull/8385#issuecomment-1936624185 merged - thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers (1.x) [nifi]
joewitt closed pull request #8385: NIFI-12769 Updated copyright year to 2024 in NOTICE file headers (1.x) URL: https://github.com/apache/nifi/pull/8385 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12769) Update Copyright for 2024
[ https://issues.apache.org/jira/browse/NIFI-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816224#comment-17816224 ] ASF subversion and git services commented on NIFI-12769: Commit b6b0b241c3a4d71454bb29228a468e40bde1d045 in nifi's branch refs/heads/main from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b6b0b241c3 ] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers This closes #8384. Signed-off-by: Joseph Witt > Update Copyright for 2024 > - > > Key: NIFI-12769 > URL: https://issues.apache.org/jira/browse/NIFI-12769 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers [nifi]
asfgit closed pull request #8384: NIFI-12769 Updated copyright year to 2024 in NOTICE file headers URL: https://github.com/apache/nifi/pull/8384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12764) Remove commons-codec and commons-lang3 from nifi-security-utils
[
https://issues.apache.org/jira/browse/NIFI-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-12764:
--
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove commons-codec and commons-lang3 from nifi-security-utils
> ---
>
> Key: NIFI-12764
> URL: https://issues.apache.org/jira/browse/NIFI-12764
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{nifi-security-utils}} module is a dependency of many other components
> and should have a minimal set of dependencies. With the introduction of Java
> HexFormat, Apache Commons Codec is no longer necessary in
> {{{}nifi-security-utils{}}}. The module also makes minimal use of Apache
> Commons Lang3, so references to {{StringUtils}} can be reduced and refactored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12768) Intermittent Failures in TestListFile.testFilterAge
[
https://issues.apache.org/jira/browse/NIFI-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816220#comment-17816220
]
ASF subversion and git services commented on NIFI-12768:
Commit e93fb17b66aea4940c076724b9ba8702172847fc in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e93fb17b66 ]
NIFI-12768 Removed unstable filename assertion in TestListFile (#8383)
> Intermittent Failures in TestListFile.testFilterAge
> ---
>
> Key: NIFI-12768
> URL: https://issues.apache.org/jira/browse/NIFI-12768
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The TestListFIle class has not changed substantively in quite some time, but
> it has begun to fail more recently across multiple platforms on GitHub Action
> runners.
> The {{testFilterAge}} method often fails with the same stack trace:
> {noformat}
> Error: org.apache.nifi.processors.standard.TestListFile.testFilterAge --
> Time elapsed: 6.436 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected: but was:
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1141)
> at
> org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:331)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> {noformat}
> The test method use recalculated timestamps to set file modification time, so
> the problem appears to be related to these timing calculations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12768) Intermittent Failures in TestListFile.testFilterAge
[
https://issues.apache.org/jira/browse/NIFI-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-12768:
--
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Intermittent Failures in TestListFile.testFilterAge
> ---
>
> Key: NIFI-12768
> URL: https://issues.apache.org/jira/browse/NIFI-12768
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The TestListFIle class has not changed substantively in quite some time, but
> it has begun to fail more recently across multiple platforms on GitHub Action
> runners.
> The {{testFilterAge}} method often fails with the same stack trace:
> {noformat}
> Error: org.apache.nifi.processors.standard.TestListFile.testFilterAge --
> Time elapsed: 6.436 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected: but was:
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1141)
> at
> org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:331)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> {noformat}
> The test method use recalculated timestamps to set file modification time, so
> the problem appears to be related to these timing calculations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12764) Remove commons-codec and commons-lang3 from nifi-security-utils
[
https://issues.apache.org/jira/browse/NIFI-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816219#comment-17816219
]
ASF subversion and git services commented on NIFI-12764:
Commit 2ea4838157748de8b597357ac7b7af4ff504d61e in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2ea4838157 ]
NIFI-12764 Removed Commons Codec and Lang3 from security-utils (#8380)
> Remove commons-codec and commons-lang3 from nifi-security-utils
> ---
>
> Key: NIFI-12764
> URL: https://issues.apache.org/jira/browse/NIFI-12764
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{nifi-security-utils}} module is a dependency of many other components
> and should have a minimal set of dependencies. With the introduction of Java
> HexFormat, Apache Commons Codec is no longer necessary in
> {{{}nifi-security-utils{}}}. The module also makes minimal use of Apache
> Commons Lang3, so references to {{StringUtils}} can be reduced and refactored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12768 Remove unstable filename assertion in TestListFile [nifi]
markap14 merged PR #8383: URL: https://github.com/apache/nifi/pull/8383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12764 Removed Commons Codec and Lang3 from security-utils [nifi]
markap14 merged PR #8380: URL: https://github.com/apache/nifi/pull/8380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers (1.x) [nifi]
turcsanyip opened a new pull request, #8385: URL: https://github.com/apache/nifi/pull/8385 # Summary [NIFI-12769](https://issues.apache.org/jira/browse/NIFI-12769) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12769) Update Copyright for 2024
[ https://issues.apache.org/jira/browse/NIFI-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12769: --- Status: Patch Available (was: Open) > Update Copyright for 2024 > - > > Key: NIFI-12769 > URL: https://issues.apache.org/jira/browse/NIFI-12769 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12769 Updated copyright year to 2024 in NOTICE file headers [nifi]
turcsanyip opened a new pull request, #8384: URL: https://github.com/apache/nifi/pull/8384 # Summary [NIFI-12769](https://issues.apache.org/jira/browse/NIFI-12769) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12769) Update Copyright for 2024
Peter Turcsanyi created NIFI-12769: -- Summary: Update Copyright for 2024 Key: NIFI-12769 URL: https://issues.apache.org/jira/browse/NIFI-12769 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12768) Intermittent Failures in TestListFile.testFilterAge
[
https://issues.apache.org/jira/browse/NIFI-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12768:
Status: Patch Available (was: Open)
> Intermittent Failures in TestListFile.testFilterAge
> ---
>
> Key: NIFI-12768
> URL: https://issues.apache.org/jira/browse/NIFI-12768
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The TestListFIle class has not changed substantively in quite some time, but
> it has begun to fail more recently across multiple platforms on GitHub Action
> runners.
> The {{testFilterAge}} method often fails with the same stack trace:
> {noformat}
> Error: org.apache.nifi.processors.standard.TestListFile.testFilterAge --
> Time elapsed: 6.436 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected: but was:
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1141)
> at
> org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:331)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> {noformat}
> The test method use recalculated timestamps to set file modification time, so
> the problem appears to be related to these timing calculations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12768 Remove unstable filename assertion in TestListFile [nifi]
exceptionfactory opened a new pull request, #8383: URL: https://github.com/apache/nifi/pull/8383 # Summary [NIFI-12768](https://issues.apache.org/jira/browse/NIFI-12768) Removes an unstable filename assertion from the `TestListFile.testFilterAge()` method. The test method uses calculated timestamps to update multiple files and then runs the `ListFile` Processor to return a filtered number of files. The `testFilterAge()` method fails intermittently when checking for which file matched, but the previous assertion succeeds, confirming that only one file matched the filter. This change is intentionally narrow, so additional changes may be necessary to revisit the larger file modification time approach if this continues to be a problem. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12768) Intermittent Failures in TestListFile.testFilterAge
David Handermann created NIFI-12768:
---
Summary: Intermittent Failures in TestListFile.testFilterAge
Key: NIFI-12768
URL: https://issues.apache.org/jira/browse/NIFI-12768
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M2
Reporter: David Handermann
Assignee: David Handermann
The TestListFIle class has not changed substantively in quite some time, but it
has begun to fail more recently across multiple platforms on GitHub Action
runners.
The {{testFilterAge}} method often fails with the same stack trace:
{noformat}
Error: org.apache.nifi.processors.standard.TestListFile.testFilterAge -- Time
elapsed: 6.436 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: but was:
at
org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at
org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1141)
at
org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:331)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
{noformat}
The test method use recalculated timestamps to set file modification time, so
the problem appears to be related to these timing calculations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11171) Reorganize Standard Components for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816187#comment-17816187
]
David Handermann commented on NIFI-11171:
-
[~dstiegli1] Thanks for the background work and summary of the various
Processors and dependencies.
On further consideration of this issue in general, I think we should hold off
further changes. The Jolt refactoring was the most obvious because of the
scattered components across different NAR bundles.
Although moving some things out of the standard NAR would be optimal, it would
have a more direct impact on existing users when upgrading.
This background is very helpful, and could serve as a basis for considering
future work on redesigned components in some cases. For now, however, unless
project members weigh in different opinion, it seems best to avoid moving some
of these more significant components.
> Reorganize Standard Components for 2.0.0
>
>
> Key: NIFI-11171
> URL: https://issues.apache.org/jira/browse/NIFI-11171
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
>
> The {{nifi-standard-processors}} and {{nifi-standard-services}} modules
> include a large number of Processors and Controller Services supporting an
> array of capabilities. Some of these capabilities require specialized
> libraries that apply to a limited number of components.
> Moving Processors and Controller Services changes class names and bundle
> coordinates which will break existing flow configurations. For this reason,
> the selection of components for reorganization should be limited and focused.
> Components with less frequent updates or usage and components with large
> dependencies trees should be considered.
> The following items should be considered as described in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals]:
> * SFTP Processors
> * Jolt Transform Processors
> * Jetty HTTP Processors
> * JSON Processors
> * Netty-based Processors
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12300) OAuth2 support in RestLookupService
[ https://issues.apache.org/jira/browse/NIFI-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory M. Foreman updated NIFI-12300: -- Issue Type: Improvement (was: New Feature) > OAuth2 support in RestLookupService > --- > > Key: NIFI-12300 > URL: https://issues.apache.org/jira/browse/NIFI-12300 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Gregory M. Foreman >Priority: Major > > Enable the RestLookupService to use StandardOauth2AccessTokenProvider to > support REST services that require oauth2 tokens to access. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12744) Solr processors do not work with Nifi 2 M2
[
https://issues.apache.org/jira/browse/NIFI-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816181#comment-17816181
]
Joe Witt commented on NIFI-12744:
-
The Apache SOLR community does appear to have an initial plan being formed to
enable moving to Jetty 12 and thus Java 17 which also likely means Spring
6/etc.. so we might be able to restore the SOLR integration quite soon in that
case.
https://issues.apache.org/jira/browse/SOLR-17069
> Solr processors do not work with Nifi 2 M2
> --
>
> Key: NIFI-12744
> URL: https://issues.apache.org/jira/browse/NIFI-12744
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andreas Koch
>Priority: Critical
>
> *steps to reproduce*
> * we launched Nifi in Docker
> * build the nar file for nifi-solr-processor
> * deploy the nar file to nifi
> * Configure Query Solr Processor for cloude
> * execute query
>
> {code:java}
> // code placeholder2024-02-06 08:47:28,886 ERROR [Timer-Driven Process
> Thread-4] o.apache.nifi.processors.solr.QuerySolr
> QuerySolr[id=7d932b59-018d-1000-404a-2744f73f9f8c] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.IncompatibleClassChangeError: class
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool$RetainedBucket has
> interface org.eclipse.jetty.util.Pool as super class
> java.lang.IncompatibleClassChangeError: class
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool$RetainedBucket has
> interface org.eclipse.jetty.util.Pool as super class at
> java.base/java.lang.ClassLoader.defineClass1(Native Method) at
> java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027) at
> java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
> at
> java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
> at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
> at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
> at
> java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) at
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool.(ArrayRetainableByteBufferPool.java:148)
> at
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool.(ArrayRetainableByteBufferPool.java:95)
> at
> org.eclipse.jetty.io.MappedByteBufferPool$Retained.(MappedByteBufferPool.java:317)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.newRetainableByteBufferPool(MappedByteBufferPool.java:140)
> at
> org.eclipse.jetty.io.AbstractByteBufferPool.(AbstractByteBufferPool.java:68)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:133)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:89)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:77)
> at org.eclipse.jetty.client.HttpClient.doStart(HttpClient.java:206)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:288)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:180)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:1099)
> at
> org.apache.nifi.processors.solr.SolrUtils.createSolrClient(SolrUtils.java:201)
> at
> org.apache.nifi.processors.solr.SolrProcessor.createSolrClient(SolrProcessor.java:153)
> at
> org.apache.nifi.processors.solr.SolrProcessor.onScheduled(SolrProcessor.java:79)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
>
[jira] [Commented] (NIFI-12738) Nifi registry ranger integration is not working
[
https://issues.apache.org/jira/browse/NIFI-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816179#comment-17816179
]
Joe Witt commented on NIFI-12738:
-
Via JIRA [this|https://issues.apache.org/jira/browse/RANGER-4038] appears to be
the most recent discussion on the topic and indeed it starts to touch on Spring
versions/Jetty (via jakarta) versions etc...
That was March 2023. No other reflections found. It sounds like they remain
committed to Java 8 at this point which in turn has the same knock-on effects
we had to consider for the NiFi community.
I recommend we pursue deprecation in 1.x/removal in 2.x and let this component
by maintained/operated by downstream builders of NiFi.
> Nifi registry ranger integration is not working
> ---
>
> Key: NIFI-12738
> URL: https://issues.apache.org/jira/browse/NIFI-12738
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
>Affects Versions: 2.0.0-M2, 2.0.0
>Reporter: Zoltán Kornél Török
>Priority: Critical
>
> h1. Bug description
> I want to try recently released nifi 2 m2 release in an environment,
> where we use ranger. Nifi registry ui was not able to load and I got the
> following error in the log:
> {code:java}
> Caused by:
> org.apache.nifi.registry.security.authorization.AuthorizerFactoryException:
> Failed to construct Authorizer.
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory.getAuthorizer(AuthorizerFactory.java:267)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.CGLIB$getAuthorizer$4()
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$FastClass$$1.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258)
> at
> org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:351)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:717)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$1.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:140)
> ... 89 common frames omitted
> Caused by:
> org.apache.nifi.registry.security.exception.SecurityProviderCreationException:
> Error creating RangerBasePlugin
> at
> org.apache.nifi.registry.ranger.RangerAuthorizer.onConfigured(RangerAuthorizer.java:178)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$ManagedAuthorizerWrapper.onConfigured(AuthorizerFactory.java:817)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory.getAuthorizer(AuthorizerFactory.java:260)
> ... 110 common frames omitted
> Caused by: java.lang.NoClassDefFoundError: javax/ws/rs/core/Cookie
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Class.java:421)
> at java.base/java.lang.Class.forName(Class.java:412)
> at
>
[jira] [Commented] (NIFI-12738) Nifi registry ranger integration is not working
[
https://issues.apache.org/jira/browse/NIFI-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816178#comment-17816178
]
Joe Witt commented on NIFI-12738:
-
Did some investigation into Ranger's mailing lists on the topic of Jetty and/or
Spring. They're on Jetty 9.x and Spring 5.x as of Nov which is the last
reflection I see. Previous attempts by dependabot to move to Jetty 10 were not
pursued. No sign of discussion there.
Will now look into JIRA or Git to see if there is any signs of changes there.
> Nifi registry ranger integration is not working
> ---
>
> Key: NIFI-12738
> URL: https://issues.apache.org/jira/browse/NIFI-12738
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
>Affects Versions: 2.0.0-M2, 2.0.0
>Reporter: Zoltán Kornél Török
>Priority: Critical
>
> h1. Bug description
> I want to try recently released nifi 2 m2 release in an environment,
> where we use ranger. Nifi registry ui was not able to load and I got the
> following error in the log:
> {code:java}
> Caused by:
> org.apache.nifi.registry.security.authorization.AuthorizerFactoryException:
> Failed to construct Authorizer.
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory.getAuthorizer(AuthorizerFactory.java:267)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.CGLIB$getAuthorizer$4()
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$FastClass$$1.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258)
> at
> org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:351)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:717)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$1.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:140)
> ... 89 common frames omitted
> Caused by:
> org.apache.nifi.registry.security.exception.SecurityProviderCreationException:
> Error creating RangerBasePlugin
> at
> org.apache.nifi.registry.ranger.RangerAuthorizer.onConfigured(RangerAuthorizer.java:178)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$ManagedAuthorizerWrapper.onConfigured(AuthorizerFactory.java:817)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory.getAuthorizer(AuthorizerFactory.java:260)
> ... 110 common frames omitted
> Caused by: java.lang.NoClassDefFoundError: javax/ws/rs/core/Cookie
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Class.java:421)
> at java.base/java.lang.Class.forName(Class.java:412)
> at
> org.apache.ranger.plugin.policyengine.RangerPluginContext.createAdminClient(RangerPluginContext.java:96)
> at
>
[jira] [Commented] (NIFI-12744) Solr processors do not work with Nifi 2 M2
[
https://issues.apache.org/jira/browse/NIFI-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816177#comment-17816177
]
Joe Witt commented on NIFI-12744:
-
Thanks for flagging/finding this issue. We need to investigate the plans of
Solr to move off these now end of life versions of Jetty (and perhaps need to
vet other libraries as well in use). If there is no near term plan we need to
deprecate our Solr processors in the 1.x line and remove them in the NiFi 2.x
line. We could restore them later should the updates end up happening but we
cannot/should not run support these end of life variants as the
dependency/vulnerability management then is highly problematic. Nothing of
course stops a downstream user of nifi maintaining them as they wish and
honoring them in their distro. Needs investigation similar to what I'm looking
into for Apache Ranger right now.
> Solr processors do not work with Nifi 2 M2
> --
>
> Key: NIFI-12744
> URL: https://issues.apache.org/jira/browse/NIFI-12744
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andreas Koch
>Priority: Critical
>
> *steps to reproduce*
> * we launched Nifi in Docker
> * build the nar file for nifi-solr-processor
> * deploy the nar file to nifi
> * Configure Query Solr Processor for cloude
> * execute query
>
> {code:java}
> // code placeholder2024-02-06 08:47:28,886 ERROR [Timer-Driven Process
> Thread-4] o.apache.nifi.processors.solr.QuerySolr
> QuerySolr[id=7d932b59-018d-1000-404a-2744f73f9f8c] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.IncompatibleClassChangeError: class
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool$RetainedBucket has
> interface org.eclipse.jetty.util.Pool as super class
> java.lang.IncompatibleClassChangeError: class
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool$RetainedBucket has
> interface org.eclipse.jetty.util.Pool as super class at
> java.base/java.lang.ClassLoader.defineClass1(Native Method) at
> java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027) at
> java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
> at
> java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
> at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
> at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
> at
> java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) at
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool.(ArrayRetainableByteBufferPool.java:148)
> at
> org.eclipse.jetty.io.ArrayRetainableByteBufferPool.(ArrayRetainableByteBufferPool.java:95)
> at
> org.eclipse.jetty.io.MappedByteBufferPool$Retained.(MappedByteBufferPool.java:317)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.newRetainableByteBufferPool(MappedByteBufferPool.java:140)
> at
> org.eclipse.jetty.io.AbstractByteBufferPool.(AbstractByteBufferPool.java:68)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:133)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:89)
> at
> org.eclipse.jetty.io.MappedByteBufferPool.(MappedByteBufferPool.java:77)
> at org.eclipse.jetty.client.HttpClient.doStart(HttpClient.java:206)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:288)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:180)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:1099)
> at
> org.apache.nifi.processors.solr.SolrUtils.createSolrClient(SolrUtils.java:201)
> at
> org.apache.nifi.processors.solr.SolrProcessor.createSolrClient(SolrProcessor.java:153)
> at
> org.apache.nifi.processors.solr.SolrProcessor.onScheduled(SolrProcessor.java:79)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> at
>
[jira] [Created] (NIFI-12767) Provenance - General error handling in API responses
Matt Gilman created NIFI-12767: -- Summary: Provenance - General error handling in API responses Key: NIFI-12767 URL: https://issues.apache.org/jira/browse/NIFI-12767 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Introduce general error handling (banner, full screen, snackbar) for APIs calls in the Provenance page. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12665) Fetch dialog for Parameter Providers
[ https://issues.apache.org/jira/browse/NIFI-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12665: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Fetch dialog for Parameter Providers > > > Key: NIFI-12665 > URL: https://issues.apache.org/jira/browse/NIFI-12665 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Implement the dialog that fetches parameters from a provider and maps them to > a parameter context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12665) Fetch dialog for Parameter Providers
[ https://issues.apache.org/jira/browse/NIFI-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816175#comment-17816175 ] ASF subversion and git services commented on NIFI-12665: Commit 7dc696ecdca1a77f78be5f6169b39b400dd60f02 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7dc696ecdc ] [NIFI-12665] fetch parameter provider parameters (#8367) * [NIFI-12665] - fetch parameter provider parameters * add permissions checks to parameter provider table actions for edit, delete, and fetch. * error handling * routing to fetch parameter provider dialog * list the parameter groups * parameter sensitivity selction/deselection * show banner error if any affected/referencing components are not readable and writeable * add indicators to parameters for changed, new, removed, ... * refactored parameter-references component to the common area. leveraged it in the fetch dialog. * validate the fetch form * submit the fetch parameter provider parameters * make the async update step completion icon color theme-aware * add missing license header * fixes for the initial round of review comments * fixing issues found in review * fix registry clients test * stop polling when there is an api error * use sort and join pipes in a couple of more places. * protect references to parameter provider in the context for read permissions * when full screen error is triggered, close any open dialog with the 'ROUTED' result to prevent unintended afterClosed actions taking place (like re-selection) * handle fetch parameter provider error * remove TODO comments * call fullScreenError correctly This closes #8367 > Fetch dialog for Parameter Providers > > > Key: NIFI-12665 > URL: https://issues.apache.org/jira/browse/NIFI-12665 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > Implement the dialog that fetches parameters from a provider and maps them to > a parameter context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12665] fetch parameter provider parameters [nifi]
mcgilman merged PR #8367: URL: https://github.com/apache/nifi/pull/8367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12738) Nifi registry ranger integration is not working
[
https://issues.apache.org/jira/browse/NIFI-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816174#comment-17816174
]
Joe Witt commented on NIFI-12738:
-
Unless we have strong indications that the Apache Ranger community plans to
update to the latest of these key components soon then for the reasons noted
above - I would encourage we deprecate the ranger nar entirely in the 1.x line
and remove from the 2.x line. Usage/maintenance of the Ranger nar can live on
in downstream users of this component.
We are facing strong counter currents as it relates to open source components
* Spring, Jetty, and many others moving forward to dependencies on Java 11, 17
and newer which has led us to move forward in NiFi to Java 21
* At the same time the vulnerability landscape for dependencies is not slowing
down.
* Meanwhile, there are many vendors that have to honor older Java version
obligations such as Java 8 that make them updating their components something
that cannot be done as quickly.
We cannot reasonably optimize for all three cases at once but we can from a
nifi community prioritize the first two.
> Nifi registry ranger integration is not working
> ---
>
> Key: NIFI-12738
> URL: https://issues.apache.org/jira/browse/NIFI-12738
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
>Affects Versions: 2.0.0-M2, 2.0.0
>Reporter: Zoltán Kornél Török
>Priority: Critical
>
> h1. Bug description
> I want to try recently released nifi 2 m2 release in an environment,
> where we use ranger. Nifi registry ui was not able to load and I got the
> following error in the log:
> {code:java}
> Caused by:
> org.apache.nifi.registry.security.authorization.AuthorizerFactoryException:
> Failed to construct Authorizer.
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory.getAuthorizer(AuthorizerFactory.java:267)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.CGLIB$getAuthorizer$4()
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$FastClass$$1.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258)
> at
> org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$0.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:351)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:717)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$$SpringCGLIB$$1.getAuthorizer()
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:140)
> ... 89 common frames omitted
> Caused by:
> org.apache.nifi.registry.security.exception.SecurityProviderCreationException:
> Error creating RangerBasePlugin
> at
> org.apache.nifi.registry.ranger.RangerAuthorizer.onConfigured(RangerAuthorizer.java:178)
> at
> org.apache.nifi.registry.security.authorization.AuthorizerFactory$ManagedAuthorizerWrapper.onConfigured(AuthorizerFactory.java:817)
>
[jira] [Updated] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
[
https://issues.apache.org/jira/browse/NIFI-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12766:
---
Status: Patch Available (was: Open)
> Fix Region handling in AssumeRoleCredentialsStrategy in
> AWSCredentialsProviderControllerService
> ---
>
> Key: NIFI-12766
> URL: https://issues.apache.org/jira/browse/NIFI-12766
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Either region or custom endpoint configuration (including the region) can be
> set on {{AWSSecurityTokenServiceClientBuilder}}.
>
> {code:java}
> 2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
> o.a.n.c.s.StandardControllerServiceNode
> StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
> name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
> Failed to invoke @OnEnabled method
> java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
> may be set.
> at
> com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
> at
> com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
> at
> com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
> at
> org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12766 Fixed Region handling in AssumeRoleCredentialsStrategy in … [nifi]
turcsanyip opened a new pull request, #8382: URL: https://github.com/apache/nifi/pull/8382 …AWSCredentialsProviderControllerService # Summary [NIFI-12766](https://issues.apache.org/jira/browse/NIFI-12766) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-4385) Adjust the QueryDatabaseTable processor for handling big tables.
[ https://issues.apache.org/jira/browse/NIFI-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816164#comment-17816164 ] Lucas Read edited comment on NIFI-4385 at 2/9/24 5:47 PM: -- [~pwicks] My understanding that the fetch size was for the data returned on the resultset not how many flowfiles are outputed. The batch setting would be more for putdatabaserecord processors. was (Author: readl1): My understanding that the fetch size was for the data returned on the resultset not how many flowfiles are outputed. > Adjust the QueryDatabaseTable processor for handling big tables. > > > Key: NIFI-4385 > URL: https://issues.apache.org/jira/browse/NIFI-4385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Tim Späth >Priority: Major > > When querying large database tables, the *QueryDatabaseTable* processor does > not perform very well. > The processor will always perform the full query and then transfer all > flowfiles as a list instead of > transferring them particularly after the *ResultSet* is fetching the next > rows(If a fetch size is given). > If you want to query a billion rows from a table, > the processor will add all flowfiles in an ArrayList in memory > before transferring the whole list after the last row is fetched by the > ResultSet. > I've checked the code in > *org.apache.nifi.processors.standard.QueryDatabaseTable.java* > and in my opinion, it would be no big deal to move the session.transfer to a > proper position in the code (into the while loop where the flowfile is added > to the list) to > achieve a real _stream support_. There was also a bug report for this problem > which resulted in adding the new property *Maximum Number of Fragments*, > but this property will just limit the results. > Now you have to multiply *Max Rows Per Flow File* with *Maximum Number of > Fragments* to get your limit, > which is not really a solution for the original problem imho. > Also the workaround with GenerateTableFetch and/or ExecuteSQL processors is > much slower than using a database cursor or a ResultSet > and stream the rows in flowfiles directly in the queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
[
https://issues.apache.org/jira/browse/NIFI-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12766:
---
Description:
Either region or custom endpoint configuration (including the region) can be
set on {{AWSSecurityTokenServiceClientBuilder}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
was:
Either region or custom endpoint configuration (including the region) can be
set on
{{{}AWSSecurityTokenServiceClientBuilder{}}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
> Fix Region handling in AssumeRoleCredentialsStrategy in
> AWSCredentialsProviderControllerService
> ---
>
> Key: NIFI-12766
> URL: https://issues.apache.org/jira/browse/NIFI-12766
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> Either region or custom endpoint configuration (including the region) can be
> set on {{AWSSecurityTokenServiceClientBuilder}}.
>
> {code:java}
> 2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
> o.a.n.c.s.StandardControllerServiceNode
> StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
> name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
> Failed to invoke @OnEnabled method
> java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
> may be set.
> at
> com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
> at
> com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
> at
> com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
> at
> org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
Peter Turcsanyi created NIFI-12766:
--
Summary: Fix Region handling in AssumeRoleCredentialsStrategy in
AWSCredentialsProviderControllerService
Key: NIFI-12766
URL: https://issues.apache.org/jira/browse/NIFI-12766
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
Either region or custom endpoint configuration (including the region) can be
set on
{{{}AWSSecurityTokenServiceClientBuilder{}}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-4385) Adjust the QueryDatabaseTable processor for handling big tables.
[ https://issues.apache.org/jira/browse/NIFI-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816164#comment-17816164 ] Lucas Read commented on NIFI-4385: -- My understanding that the fetch size was for the data returned on the resultset not how many flowfiles are outputed. > Adjust the QueryDatabaseTable processor for handling big tables. > > > Key: NIFI-4385 > URL: https://issues.apache.org/jira/browse/NIFI-4385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Tim Späth >Priority: Major > > When querying large database tables, the *QueryDatabaseTable* processor does > not perform very well. > The processor will always perform the full query and then transfer all > flowfiles as a list instead of > transferring them particularly after the *ResultSet* is fetching the next > rows(If a fetch size is given). > If you want to query a billion rows from a table, > the processor will add all flowfiles in an ArrayList in memory > before transferring the whole list after the last row is fetched by the > ResultSet. > I've checked the code in > *org.apache.nifi.processors.standard.QueryDatabaseTable.java* > and in my opinion, it would be no big deal to move the session.transfer to a > proper position in the code (into the while loop where the flowfile is added > to the list) to > achieve a real _stream support_. There was also a bug report for this problem > which resulted in adding the new property *Maximum Number of Fragments*, > but this property will just limit the results. > Now you have to multiply *Max Rows Per Flow File* with *Maximum Number of > Fragments* to get your limit, > which is not really a solution for the original problem imho. > Also the workaround with GenerateTableFetch and/or ExecuteSQL processors is > much slower than using a database cursor or a ResultSet > and stream the rows in flowfiles directly in the queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-4385) Adjust the QueryDatabaseTable processor for handling big tables.
[ https://issues.apache.org/jira/browse/NIFI-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816151#comment-17816151 ] Peter Wicks commented on NIFI-4385: --- Hi [~readl1] . If before your flow was expecting all rows in a single FlowFile, it would get multiple flow files after this change, based on batch size. So if you used to have a single 100k row FlowFile once an hour, now you would get 100 flow files per hour. Depending on the workflow, that could be a big deal. I think the important thing is that you can do what you want to do, it just requires a config change at the processor level based on your needs. > Adjust the QueryDatabaseTable processor for handling big tables. > > > Key: NIFI-4385 > URL: https://issues.apache.org/jira/browse/NIFI-4385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Tim Späth >Priority: Major > > When querying large database tables, the *QueryDatabaseTable* processor does > not perform very well. > The processor will always perform the full query and then transfer all > flowfiles as a list instead of > transferring them particularly after the *ResultSet* is fetching the next > rows(If a fetch size is given). > If you want to query a billion rows from a table, > the processor will add all flowfiles in an ArrayList in memory > before transferring the whole list after the last row is fetched by the > ResultSet. > I've checked the code in > *org.apache.nifi.processors.standard.QueryDatabaseTable.java* > and in my opinion, it would be no big deal to move the session.transfer to a > proper position in the code (into the while loop where the flowfile is added > to the list) to > achieve a real _stream support_. There was also a bug report for this problem > which resulted in adding the new property *Maximum Number of Fragments*, > but this property will just limit the results. > Now you have to multiply *Max Rows Per Flow File* with *Maximum Number of > Fragments* to get your limit, > which is not really a solution for the original problem imho. > Also the workaround with GenerateTableFetch and/or ExecuteSQL processors is > much slower than using a database cursor or a ResultSet > and stream the rows in flowfiles directly in the queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] MINIFICPP-2283 Create tool to encrypt sensitive properties in config.yml [nifi-minifi-cpp]
fgerlits opened a new pull request, #1725: URL: https://github.com/apache/nifi-minifi-cpp/pull/1725 https://issues.apache.org/jira/browse/MINIFICPP-2283 I have also upgraded argparse from 2.9 to 3.0 so I can use the new `flag()` syntax. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12765: Ranger audit reporting work only with jetty 11.x version [nifi]
taz1988 opened a new pull request, #8381: URL: https://github.com/apache/nifi/pull/8381 # Summary [NIFI-12765](https://issues.apache.org/jira/browse/NIFI-12765) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-12765) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12765) Nifi and nifi registry ranger audit is broken
Zoltán Kornél Török created NIFI-12765:
--
Summary: Nifi and nifi registry ranger audit is broken
Key: NIFI-12765
URL: https://issues.apache.org/jira/browse/NIFI-12765
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M2, 2.0.0
Reporter: Zoltán Kornél Török
Assignee: Zoltán Kornél Török
h3. Bug description
Currently ranger plugins are not reporting audit events into ranger.
h2. Investigation
In the nifi log I found the following ("classic") NoClassDefFoundError:
{code:java}
ERROR org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect
to Solr server.
ZooKeepers=cfm-oudjal-dd-master0.cfm-5pax.svbr-nqvp.int.cldr.work:2181/solr-infrajava.lang.NoClassDefFoundError:
org/eclipse/jetty/client/util/SPNEGOAuthentication
at
org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:168)
at
org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:227)
at
org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
at
org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
... 5 common frames omitted {code}
As you can see ranger-audit depends on solr client which depends on jetty
client.
The problem is that solr client class use
org.eclipse.jetty.client.util.SPNEGOAuthentication -
[https://github.infra.cloudera.com/CDH/solr/blob/solr9-master/solr/solrj/src/java/org/apache/solr/client/solrj/impl/Krb5HttpClientBuilder.java#L46]
However in case jetty 12.x line, this class is moved to another package:
[https://github.com/jetty/jetty.project/commit/a1c5cefd0d5657df04e5364cca9315aa4e2a1aef]
So the problem exist, since jetty version upgraded to 12
h2. Proposed solution
Sadly there is no available solr client (or ranger client), which haven't had
this dependency. The only solution what I found (and propose in my pr) is to
override jetty version in case of ranger plugins to jetty line 11, where this
class is not moved. I tested it on my environment and the audit logging to
ranger worked well with that version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [NIFI-12758] create remote process group [nifi]
mcgilman commented on code in PR #8376:
URL: https://github.com/apache/nifi/pull/8376#discussion_r1484505949
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -222,6 +224,17 @@ export class FlowEffects {
}),
catchError((error) =>
of(FlowActions.flowApiError({ error: error.error })))
);
+case ComponentType.RemoteProcessGroup:
+return
from(this.flowService.getParameterContexts()).pipe(
Review Comment:
The create Remote Process Group dialog does not require the Parameter
Contexts. This `case` should be able to simply return the action for opening
the dialog. Kind of like the cases for Processors or Ports.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/ui/canvas/items/remote-process-group/create-remote-process-group/create-remote-process-group.component.html:
##
@@ -0,0 +1,115 @@
+
+
+Create Remote Process Group
+
+
+
+
+
+
+URLs
+https://remotehost:8443/nifi; />
+
+
+
+
+
+
+Transport Protocol
+
+
+RAW
+
+
+HTTP
+
+
+
+
+
+
+Local Network Interface
+
+
+
+
+
+
+
+HTTP Proxy Server Hostname
+
+
+
+
+
+HTTP Proxy Server Port
+
+
+
+
+
+
+
+HTTP Proxy User
+
+
+
+
+
+HTTP Proxy Password
+
+
+
+
+
+
+
+Communications Timeout
+
+
+
+
+
+Yield Duration
+
+
+
+
+
+
+Cancel
+
+Add
+
+
Review Comment:
Can these buttons be aligned to the bottom? In this case, this appears to be
happening because the dialog content does not fill the space of the panel class
that was chosen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12746) BUG - Search closes when user attempts to put focus back in search box
[ https://issues.apache.org/jira/browse/NIFI-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12746: --- Resolution: Fixed Status: Resolved (was: Patch Available) > BUG - Search closes when user attempts to put focus back in search box > -- > > Key: NIFI-12746 > URL: https://issues.apache.org/jira/browse/NIFI-12746 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Scott Aslan >Priority: Major > Attachments: Kapture 2024-02-06 at 10.29.59.gif > > Time Spent: 0.5h > Remaining Estimate: 0h > > If you search for something, then select a result, then attempt to place the > focus back in the search box to modify the search term, it closes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12746) BUG - Search closes when user attempts to put focus back in search box
[ https://issues.apache.org/jira/browse/NIFI-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816117#comment-17816117 ] ASF subversion and git services commented on NIFI-12746: Commit 16eadc88da1d95660b9f70fd4594e6af88513033 in nifi's branch refs/heads/main from Scott Aslan [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=16eadc88da ] [NIFI-12746] keep search field open on backdrop click (#8372) This closes #8372 > BUG - Search closes when user attempts to put focus back in search box > -- > > Key: NIFI-12746 > URL: https://issues.apache.org/jira/browse/NIFI-12746 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Scott Aslan >Priority: Major > Attachments: Kapture 2024-02-06 at 10.29.59.gif > > Time Spent: 0.5h > Remaining Estimate: 0h > > If you search for something, then select a result, then attempt to place the > focus back in the search box to modify the search term, it closes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-4385) Adjust the QueryDatabaseTable processor for handling big tables.
[ https://issues.apache.org/jira/browse/NIFI-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816118#comment-17816118 ] Lucas Read commented on NIFI-4385: -- Adding some thoughts here based on some issues we have experienced with some sql based processors over the past 2 weeks. The default configuration for SQL-based processors (ExecuteSQL, QueryDatabaseRecord, PutDatabaseRecord) in Apache NiFi currently sets the fetch/batch size to 0. This default configuration can lead to significant heap impacts with Java when querying or inserting large amounts of records. To mitigate these impacts and improve performance, it's recommended to adjust the default configuration to a more appropriate value, such as 100 or 1000. *Suggested Changes:* * Update the default configuration for the following SQL-based processors (Im sure I am missing some) : ** *ExecuteSQL:* Set the default fetch/batch size to 100 or 1000. ** *QueryDatabaseRecord:* Set the default fetch/batch size to 100 or 1000. ** *PutDatabaseRecord:* Set the default fetch/batch size to 100 or 1000. *Rationale:* The current default configuration of 0 for fetch/batch size can lead to inefficient memory usage and potential heap issues, especially when dealing with large datasets. By adjusting the default configuration to a more suitable value, such as 100 or 1000, users can experience improved performance and reduced risk of memory-related errors. *Impact Analysis:* * *Performance:* Updating the default configuration is expected to improve performance for users dealing with large datasets, as it will optimize memory usage and reduce the likelihood of heap-related issues. * *Compatibility:* These changes should not impact existing workflows or configurations adversely, as they are focused on improving default settings rather than altering behavior. > Adjust the QueryDatabaseTable processor for handling big tables. > > > Key: NIFI-4385 > URL: https://issues.apache.org/jira/browse/NIFI-4385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Tim Späth >Priority: Major > > When querying large database tables, the *QueryDatabaseTable* processor does > not perform very well. > The processor will always perform the full query and then transfer all > flowfiles as a list instead of > transferring them particularly after the *ResultSet* is fetching the next > rows(If a fetch size is given). > If you want to query a billion rows from a table, > the processor will add all flowfiles in an ArrayList in memory > before transferring the whole list after the last row is fetched by the > ResultSet. > I've checked the code in > *org.apache.nifi.processors.standard.QueryDatabaseTable.java* > and in my opinion, it would be no big deal to move the session.transfer to a > proper position in the code (into the while loop where the flowfile is added > to the list) to > achieve a real _stream support_. There was also a bug report for this problem > which resulted in adding the new property *Maximum Number of Fragments*, > but this property will just limit the results. > Now you have to multiply *Max Rows Per Flow File* with *Maximum Number of > Fragments* to get your limit, > which is not really a solution for the original problem imho. > Also the workaround with GenerateTableFetch and/or ExecuteSQL processors is > much slower than using a database cursor or a ResultSet > and stream the rows in flowfiles directly in the queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12746] fix search UX bug [nifi]
mcgilman merged PR #8372: URL: https://github.com/apache/nifi/pull/8372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-12746] fix search UX bug [nifi]
mcgilman commented on PR #8372: URL: https://github.com/apache/nifi/pull/8372#issuecomment-1936142577 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12680) DefaultedDynamicClassPathModificationIT Fails due to Missing JAR
[ https://issues.apache.org/jira/browse/NIFI-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12680: Fix Version/s: 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > DefaultedDynamicClassPathModificationIT Fails due to Missing JAR > > > Key: NIFI-12680 > URL: https://issues.apache.org/jira/browse/NIFI-12680 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.26.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The system test DefaultedDynamicClassPathModificationIT fails while waiting > for a valid Processor due to a reference to commons-lang3 version 3.13.0, > which is no longer current as of recent upgrades to version 3.14.0. This test > succeeds on the current main branch, but fails on the support branch due to > the version change in the library used for testing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12680) DefaultedDynamicClassPathModificationIT Fails due to Missing JAR
[ https://issues.apache.org/jira/browse/NIFI-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816094#comment-17816094 ] ASF subversion and git services commented on NIFI-12680: Commit 6612077b42e8c483db57d3c7a1797d447b447b8d in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6612077b42 ] NIFI-12680 Fixed JAR for DefaultedDynamicClassPathModificationIT - Adjusted test flow to find available version of commons-lang3 instead of depending on a specific version from the default property value This closes #8307 Signed-off-by: David Handermann > DefaultedDynamicClassPathModificationIT Fails due to Missing JAR > > > Key: NIFI-12680 > URL: https://issues.apache.org/jira/browse/NIFI-12680 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > The system test DefaultedDynamicClassPathModificationIT fails while waiting > for a valid Processor due to a reference to commons-lang3 version 3.13.0, > which is no longer current as of recent upgrades to version 3.14.0. This test > succeeds on the current main branch, but fails on the support branch due to > the version change in the library used for testing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12680 Fix JAR for DefaultedDynamicClassPathModificationIT on Support Branch [nifi]
exceptionfactory closed pull request #8307: NIFI-12680 Fix JAR for DefaultedDynamicClassPathModificationIT on Support Branch URL: https://github.com/apache/nifi/pull/8307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12736) Fix migration of credential properties in AWS processors
[
https://issues.apache.org/jira/browse/NIFI-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12736:
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix migration of credential properties in AWS processors
>
>
> Key: NIFI-12736
> URL: https://issues.apache.org/jira/browse/NIFI-12736
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In AWS processors the processor level credential properties were removed in
> 2.0 in favour of AWS Credentials Provider Service. Migration code was added
> in NIFI-12220.
> The 1.x processors allowed to configure both the processor level Access Key
> ID + Secret Access Key and AWS Credentials Provider Service properties
> ({{{}customValidate(){}}} did not check it for some reason). In this case the
> controller service took precedence.
> The migration process first checks if Access Key ID + Secret Access Key are
> configured and if they are, it creates a new controller service even if there
> was another one already in use (and the new may have outdated migrated
> credential values as the pre-existing service was used effectively in the
> flow).
> Modify the migration logic to avoid controller service duplication and to use
> the existing service.
> Also modify the default credential to
> [anonymous|https://github.com/apache/nifi/blob/ce56117e43c7cdce37ca76d54550a2b45acb508a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L415]
> as it was used if no properties were configured at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12736 Fixed migration of credential properties in AWS processors [nifi]
exceptionfactory closed pull request #8353: NIFI-12736 Fixed migration of credential properties in AWS processors URL: https://github.com/apache/nifi/pull/8353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12736) Fix migration of credential properties in AWS processors
[
https://issues.apache.org/jira/browse/NIFI-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816092#comment-17816092
]
ASF subversion and git services commented on NIFI-12736:
Commit d5206b1ddd6ab9be80c24fce3d9fb7ca8db7f26b in nifi's branch
refs/heads/main from Peter Turcsanyi
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d5206b1ddd ]
NIFI-12736 Fixed migration of credential properties in AWS processors
This closes #8353
Signed-off-by: David Handermann
> Fix migration of credential properties in AWS processors
>
>
> Key: NIFI-12736
> URL: https://issues.apache.org/jira/browse/NIFI-12736
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In AWS processors the processor level credential properties were removed in
> 2.0 in favour of AWS Credentials Provider Service. Migration code was added
> in NIFI-12220.
> The 1.x processors allowed to configure both the processor level Access Key
> ID + Secret Access Key and AWS Credentials Provider Service properties
> ({{{}customValidate(){}}} did not check it for some reason). In this case the
> controller service took precedence.
> The migration process first checks if Access Key ID + Secret Access Key are
> configured and if they are, it creates a new controller service even if there
> was another one already in use (and the new may have outdated migrated
> credential values as the pre-existing service was used effectively in the
> flow).
> Modify the migration logic to avoid controller service duplication and to use
> the existing service.
> Also modify the default credential to
> [anonymous|https://github.com/apache/nifi/blob/ce56117e43c7cdce37ca76d54550a2b45acb508a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L415]
> as it was used if no properties were configured at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12742) Users, Summary, Queue Listing - General error handling in API responses
[ https://issues.apache.org/jira/browse/NIFI-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12742: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Users, Summary, Queue Listing - General error handling in API responses > --- > > Key: NIFI-12742 > URL: https://issues.apache.org/jira/browse/NIFI-12742 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Introduce error handling to web requests on the Users, Summary, Queue Listing > page. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12742) Users, Summary, Queue Listing - General error handling in API responses
[ https://issues.apache.org/jira/browse/NIFI-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816089#comment-17816089 ] ASF subversion and git services commented on NIFI-12742: Commit 439c59e73362e002e8576580e9ad3f3a696ffaae in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=439c59e733 ] NIFI-12742: Error Handling in Summary, Users, and Queue Listing (#8366) - Error handling in Users. - Error handling in Summary. - Error handling in Queue Listing. - Addressing review feedback. - Dispatching delete success to ensure the active request is reset upon completion. This closes #8366 > Users, Summary, Queue Listing - General error handling in API responses > --- > > Key: NIFI-12742 > URL: https://issues.apache.org/jira/browse/NIFI-12742 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Introduce error handling to web requests on the Users, Summary, Queue Listing > page. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12740) Python Processors sometimes stuck in invalid state: 'Initializing runtime environment'
[
https://issues.apache.org/jira/browse/NIFI-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12740:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Python Processors sometimes stuck in invalid state: 'Initializing runtime
> environment'
> --
>
> Key: NIFI-12740
> URL: https://issues.apache.org/jira/browse/NIFI-12740
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 2.0.0-M1, 2.0.0-M2
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When creating a Python processor, sometimes the Processor remains in an
> invalid state with the message "Initializing runtime environment"
> In the logs, we see the following error/stack trace:
> {code:java}
> 2024-02-05 17:23:30,308 ERROR [Initialize SetRecordField]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> VirtualThread[#123,Initialize
> SetRecordField]/runnable@ForkJoinPool-1-worker-5:
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> at
> org.apache.nifi.py4j.StandardPythonBridge.findExtensionId(StandardPythonBridge.java:322)
> at
> org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:99)
> at
> org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$3(StandardPythonBridge.java:142)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:73)
> at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12742: Error Handling in Summary, Users, and Queue Listing [nifi]
rfellows merged PR #8366: URL: https://github.com/apache/nifi/pull/8366 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12740) Python Processors sometimes stuck in invalid state: 'Initializing runtime environment'
[
https://issues.apache.org/jira/browse/NIFI-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816087#comment-17816087
]
ASF subversion and git services commented on NIFI-12740:
Commit 4094c7f59940b628ac268921b64971bfbc148e39 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4094c7f599 ]
NIFI-12740 Fixed Threading Bug with Java to Python Bound Objects
Fixed a threading bug where Java makes a call to Python, Python create a Java
object to return (ArrayList), Python returns the object and then cleans it up,
notifying Java to remove it from 'bound objects', and then Java process the
response and returns a null object because it's no longer bound
This closes #8356
Signed-off-by: David Handermann
> Python Processors sometimes stuck in invalid state: 'Initializing runtime
> environment'
> --
>
> Key: NIFI-12740
> URL: https://issues.apache.org/jira/browse/NIFI-12740
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 2.0.0-M1, 2.0.0-M2
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When creating a Python processor, sometimes the Processor remains in an
> invalid state with the message "Initializing runtime environment"
> In the logs, we see the following error/stack trace:
> {code:java}
> 2024-02-05 17:23:30,308 ERROR [Initialize SetRecordField]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> VirtualThread[#123,Initialize
> SetRecordField]/runnable@ForkJoinPool-1-worker-5:
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> java.lang.NullPointerException: Cannot invoke "java.util.List.stream()"
> because "processorTypes" is null
> at
> org.apache.nifi.py4j.StandardPythonBridge.findExtensionId(StandardPythonBridge.java:322)
> at
> org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:99)
> at
> org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$3(StandardPythonBridge.java:142)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:73)
> at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12740: Fixed a threading bug where Java makes a call to Python, … [nifi]
exceptionfactory closed pull request #8356: NIFI-12740: Fixed a threading bug where Java makes a call to Python, … URL: https://github.com/apache/nifi/pull/8356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12696) Fix authorization issues when requesting FlowAnalysisResults
[ https://issues.apache.org/jira/browse/NIFI-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12696. - Fix Version/s: 2.0.0 Resolution: Fixed > Fix authorization issues when requesting FlowAnalysisResults > > > Key: NIFI-12696 > URL: https://issues.apache.org/jira/browse/NIFI-12696 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When requesting FlowAnalysisResults the authorization logic performed has a > couple of issues: > # Doesn't handle exceptions thrown when the a component producing a result is > tested to be a Port. The logic goes through possible component types and when > reaches Ports it throws an exception. > # As the logic goest through possible components, the mismatching ones throw > ResourceNotFoundExceptions. These are captured but this is a bad practice in > general. Throwing and capturing exceptions in non-exceptional cases is bad > from both design and performance perspective. > # The number of possible components checked is too limited. If a component is > unrecognized, the corresponding violation will have a PermissionDTO attached > with canRead and canWrite set to false, essentially rendering the result > unavailable and thus leading to a false negative. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12696) Fix authorization issues when requesting FlowAnalysisResults
[ https://issues.apache.org/jira/browse/NIFI-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816083#comment-17816083 ] ASF subversion and git services commented on NIFI-12696: Commit 49e599385d2527f8a05b929e00221b764b310eab in nifi's branch refs/heads/main from Tamas Palfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=49e599385d ] NIFI-12696 Added Component Type to Rule Violations for Authorization RuleViolations (these objects only reside in memory only) now contain the type of the component that is responsible for the violation. This is used in StandardNiFiServiceFacade to fix and improve the authorization logic. This closes #8318 Signed-off-by: David Handermann > Fix authorization issues when requesting FlowAnalysisResults > > > Key: NIFI-12696 > URL: https://issues.apache.org/jira/browse/NIFI-12696 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When requesting FlowAnalysisResults the authorization logic performed has a > couple of issues: > # Doesn't handle exceptions thrown when the a component producing a result is > tested to be a Port. The logic goes through possible component types and when > reaches Ports it throws an exception. > # As the logic goest through possible components, the mismatching ones throw > ResourceNotFoundExceptions. These are captured but this is a bad practice in > general. Throwing and capturing exceptions in non-exceptional cases is bad > from both design and performance perspective. > # The number of possible components checked is too limited. If a component is > unrecognized, the corresponding violation will have a PermissionDTO attached > with canRead and canWrite set to false, essentially rendering the result > unavailable and thus leading to a false negative. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12696 Fix authorization issues when requesting FlowAnalysisResults [nifi]
exceptionfactory closed pull request #8318: NIFI-12696 Fix authorization issues when requesting FlowAnalysisResults URL: https://github.com/apache/nifi/pull/8318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12710) PutDatabaseRecord processor does not handle microsecond timestamps properly
[ https://issues.apache.org/jira/browse/NIFI-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12710: Resolution: Fixed Status: Resolved (was: Patch Available) > PutDatabaseRecord processor does not handle microsecond timestamps properly > --- > > Key: NIFI-12710 > URL: https://issues.apache.org/jira/browse/NIFI-12710 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12710) PutDatabaseRecord processor does not handle microsecond timestamps properly
[ https://issues.apache.org/jira/browse/NIFI-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816080#comment-17816080 ] ASF subversion and git services commented on NIFI-12710: Commit 5f534dcc42e99a87dd8315d0d424a3b615018cd4 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5f534dcc42 ] NIFI-12710 Support microsecond precision for Timestamp Record fields - PutDatabaseRecordIT supports operating systems with either nanosecond or microsecond precision This closes #8332 Signed-off-by: David Handermann > PutDatabaseRecord processor does not handle microsecond timestamps properly > --- > > Key: NIFI-12710 > URL: https://issues.apache.org/jira/browse/NIFI-12710 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12710: Allow for microsecond precision timestamps when pushing t… [nifi]
exceptionfactory closed pull request #8332: NIFI-12710: Allow for microsecond precision timestamps when pushing t… URL: https://github.com/apache/nifi/pull/8332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update minifi-java-agent-quick-start.md [nifi]
exceptionfactory commented on PR #8379: URL: https://github.com/apache/nifi/pull/8379#issuecomment-1936000104 Thanks for the documentation improvements @wrender. For tracking purposes, all changes need an associated Apache NiFi Jira issue, as listed in the pull request checklist. You can request an account using the following link: https://selfserve.apache.org/jira-account.html After creating a Jira issue, simply update the pull request title with the Jira issue number. If you have any questions, feel free to ask! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12764) Remove commons-codec and commons-lang3 from nifi-security-utils
[
https://issues.apache.org/jira/browse/NIFI-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12764:
Status: Patch Available (was: Open)
> Remove commons-codec and commons-lang3 from nifi-security-utils
> ---
>
> Key: NIFI-12764
> URL: https://issues.apache.org/jira/browse/NIFI-12764
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{nifi-security-utils}} module is a dependency of many other components
> and should have a minimal set of dependencies. With the introduction of Java
> HexFormat, Apache Commons Codec is no longer necessary in
> {{{}nifi-security-utils{}}}. The module also makes minimal use of Apache
> Commons Lang3, so references to {{StringUtils}} can be reduced and refactored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12764 Removed Commons Codec and Lang3 from security-utils [nifi]
exceptionfactory opened a new pull request, #8380: URL: https://github.com/apache/nifi/pull/8380 # Summary [NIFI-12764](https://issues.apache.org/jira/browse/NIFI-12764) Removes `commons-codec` and `commons-lang3` dependencies from `nifi-security-utils` to reduce the transitive dependencies inherited in other referencing modules. Changes include replacing Commons Codec Hex with Java [HexFormat](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/HexFormat.html) and replacing `StringUtils` references with simple checks. Commons Codec is a runtime dependency of OpenSAML, so it is now declared explicitly in `nifi-web-security`, and Commons Lang3 is a runtime dependency of the `nifi-system-test-suite` for dynamic class loading tests. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12764) Remove commons-codec and commons-lang3 from nifi-security-utils
David Handermann created NIFI-12764:
---
Summary: Remove commons-codec and commons-lang3 from
nifi-security-utils
Key: NIFI-12764
URL: https://issues.apache.org/jira/browse/NIFI-12764
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: David Handermann
Assignee: David Handermann
The {{nifi-security-utils}} module is a dependency of many other components and
should have a minimal set of dependencies. With the introduction of Java
HexFormat, Apache Commons Codec is no longer necessary in
{{{}nifi-security-utils{}}}. The module also makes minimal use of Apache
Commons Lang3, so references to {{StringUtils}} can be reduced and refactored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12763) Theme the mat-paginator
[ https://issues.apache.org/jira/browse/NIFI-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12763: --- Attachment: Screenshot 2024-02-09 at 8.27.39 AM.png > Theme the mat-paginator > --- > > Key: NIFI-12763 > URL: https://issues.apache.org/jira/browse/NIFI-12763 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-02-08 at 11.50.08 AM.png, Screenshot > 2024-02-09 at 8.27.39 AM.png > > > The paginator we use in a couple of places isn't picking up the theming. > > > !Screenshot 2024-02-08 at 11.50.08 AM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12763) Theme the mat-paginator
[ https://issues.apache.org/jira/browse/NIFI-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12763: --- Description: The paginator we use in a couple of places isn't picking up the theming. !Screenshot 2024-02-09 at 8.27.39 AM.png|width=999,height=553! was: The paginator we use in a couple of places isn't picking up the theming. !Screenshot 2024-02-08 at 11.50.08 AM.png! > Theme the mat-paginator > --- > > Key: NIFI-12763 > URL: https://issues.apache.org/jira/browse/NIFI-12763 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-02-08 at 11.50.08 AM.png, Screenshot > 2024-02-09 at 8.27.39 AM.png > > > The paginator we use in a couple of places isn't picking up the theming. > > > !Screenshot 2024-02-09 at 8.27.39 AM.png|width=999,height=553! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12763) Theme the mat-paginator
[ https://issues.apache.org/jira/browse/NIFI-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12763: --- Description: The paginator we use in a couple of places isn't picking up the theming. was: The paginator we use in a couple of places isn't picking up the theming. > Theme the mat-paginator > --- > > Key: NIFI-12763 > URL: https://issues.apache.org/jira/browse/NIFI-12763 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-02-08 at 11.50.08 AM.png > > > The paginator we use in a couple of places isn't picking up the theming. > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12763) Theme the mat-paginator
[ https://issues.apache.org/jira/browse/NIFI-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12763: --- Description: The paginator we use in a couple of places isn't picking up the theming. !Screenshot 2024-02-08 at 11.50.08 AM.png! was: The paginator we use in a couple of places isn't picking up the theming. > Theme the mat-paginator > --- > > Key: NIFI-12763 > URL: https://issues.apache.org/jira/browse/NIFI-12763 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-02-08 at 11.50.08 AM.png > > > The paginator we use in a couple of places isn't picking up the theming. > > > !Screenshot 2024-02-08 at 11.50.08 AM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12763) Theme the mat-paginator
[ https://issues.apache.org/jira/browse/NIFI-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12763: --- Attachment: Screenshot 2024-02-08 at 11.50.08 AM.png > Theme the mat-paginator > --- > > Key: NIFI-12763 > URL: https://issues.apache.org/jira/browse/NIFI-12763 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-02-08 at 11.50.08 AM.png > > > The paginator we use in a couple of places isn't picking up the theming. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12763) Theme the mat-paginator
Rob Fellows created NIFI-12763: -- Summary: Theme the mat-paginator Key: NIFI-12763 URL: https://issues.apache.org/jira/browse/NIFI-12763 Project: Apache NiFi Issue Type: Sub-task Reporter: Rob Fellows The paginator we use in a couple of places isn't picking up the theming. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12671 Added AwsFileResourceService [nifi]
mark-bathori commented on code in PR #8368:
URL: https://github.com/apache/nifi/pull/8368#discussion_r1484280873
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/util/RegionUtilV1.java:
##
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.util;
+
+import com.amazonaws.regions.Region;
+import com.amazonaws.regions.Regions;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.processor.exception.ProcessException;
+
+import java.util.Map;
+
+import static org.apache.nifi.processors.aws.s3.AbstractS3Processor.S3_REGION;
+
+/**
+ * Utility class for AWS region methods. This class uses AWS SDK v1.
+ *
+ */
+public abstract class RegionUtilV1 {
+
+public static final String S3_REGION_ATTRIBUTE = "s3.region" ;
+static final AllowableValue ATTRIBUTE_DEFINED_REGION = new
AllowableValue("attribute-defined-region",
Review Comment:
This attribute is still duplicated, region related parts should be removed
from `AbstractS3Processor` and moved here.
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/util/RegionUtilV1.java:
##
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.util;
+
+import com.amazonaws.regions.Region;
+import com.amazonaws.regions.Regions;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.processor.exception.ProcessException;
+
+import java.util.Map;
+
+import static org.apache.nifi.processors.aws.s3.AbstractS3Processor.S3_REGION;
+
+/**
+ * Utility class for AWS region methods. This class uses AWS SDK v1.
+ *
+ */
+public abstract class RegionUtilV1 {
Review Comment:
In my opinion it is better to declare Util classes as final class with
private constructor. Abstract class provides the same restriction but it could
be misleading since the abstract keyword usually means that the class is
intended to be extended or subclassed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12672 Added Azure specific versions of FileResourceService [nifi]
mark-bathori commented on code in PR #8359:
URL: https://github.com/apache/nifi/pull/8359#discussion_r1484035099
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/services/azure/storage/AzureBlobStorageFileResourceService.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.azure.storage;
+
+import com.azure.storage.blob.BlobClient;
+import com.azure.storage.blob.BlobContainerClient;
+import com.azure.storage.blob.BlobServiceClient;
+import com.azure.storage.blob.models.BlobStorageException;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.documentation.UseCase;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.fileresource.service.api.FileResource;
+import org.apache.nifi.fileresource.service.api.FileResourceService;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processors.azure.AbstractAzureBlobProcessor_v12;
+import org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12;
+import org.apache.nifi.processors.azure.storage.utils.AzureStorageUtils;
+import org.apache.nifi.processors.azure.storage.utils.BlobServiceClientFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+
+import static
org.apache.nifi.processors.azure.storage.utils.AzureStorageUtils.getProxyOptions;
+import static
org.apache.nifi.processors.azure.storage.utils.BlobAttributes.ATTR_NAME_BLOBNAME;
+import static
org.apache.nifi.processors.azure.storage.utils.BlobAttributes.ATTR_NAME_CONTAINER;
+import static org.apache.nifi.util.StringUtils.isBlank;
+
+@Tags({"azure", "microsoft", "cloud", "storage", "file", "resource", "blob"})
+@SeeAlso({FetchAzureBlobStorage_v12.class})
+@CapabilityDescription("Provides an Azure Blob Storage file resource for other
components.")
+@UseCase(
+description = "Fetch a specific file from Azure Blob Storage." +
+" The service provides higher performance compared to fetch
processors when the data should be moved between different storages without any
transformation.",
+configuration = """
+"Container Name" = "${azure.container}"
+"Blob Name" = "${azure.blobname}"
+
+The "Storage Credentials" property should specify an instance
of the AzureStorageCredentialsService_v12 in order to provide credentials for
accessing the storage container.
+"""
+)
+public class AzureBlobStorageFileResourceService extends
AbstractControllerService implements FileResourceService {
+
+public static final PropertyDescriptor CONTAINER = new
PropertyDescriptor.Builder()
+.fromPropertyDescriptor(AzureStorageUtils.CONTAINER)
+.defaultValue(String.format("${%s}", ATTR_NAME_CONTAINER))
+.build();
+
+public static final PropertyDescriptor BLOB_NAME = new
PropertyDescriptor.Builder()
+.fromPropertyDescriptor(AbstractAzureBlobProcessor_v12.BLOB_NAME)
+.defaultValue(String.format("${%s}", ATTR_NAME_BLOBNAME))
+.build();
+
+public static final PropertyDescriptor STORAGE_CREDENTIALS_SERVICE = new
PropertyDescriptor.Builder()
+.name("storage-credentials-service")
+.displayName("Storage Credentials")
+.description("Controller Service used to obtain Azure Blob Storage
Credentials.")
+
.identifiesControllerService(AzureStorageCredentialsService_v12.class)
+.required(true)
+.build();
+
+private static final List PROPERTIES = List.of(
+CONTAINER,
+BLOB_NAME,
+STORAGE_CREDENTIALS_SERVICE
+);
+
+private volatile BlobServiceClientFactory clientFactory;
+private volatile
[jira] [Assigned] (MINIFICPP-2293) Support installing python dependencies defined inline
[ https://issues.apache.org/jira/browse/MINIFICPP-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi reassigned MINIFICPP-2293: Assignee: Gábor Gyimesi > Support installing python dependencies defined inline > - > > Key: MINIFICPP-2293 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2293 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > > In NiFi python processors python dependencies can be defined inside the class > definition in the ProcessorDetails nested class using the dependencies > attribute. The dependencies attribute is a list with the required python > packages the processor depends on. MiNiFi should also support installing the > required packages defined here. -- This message was sent by Atlassian Jira (v8.20.10#820010)
