[jira] [Commented] (NIFI-9017) If a node's load balance host/port is updated, other nodes stop sending data
[ https://issues.apache.org/jira/browse/NIFI-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394501#comment-17394501 ] Jens M Kofoed commented on NIFI-9017: - Changing hostname localhost to ip 127.0.0.1 is the same. In both cases it will only listen for connection from the localhost and other nodes will not be able to connect. Use the real ip address for the node which the other nodes can reach it at > If a node's load balance host/port is updated, other nodes stop sending data > > > Key: NIFI-9017 > URL: https://issues.apache.org/jira/browse/NIFI-9017 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > I have a 3-node cluster. On one of the nodes, I stopped it, changed the load > balance host from localhost to 127.0.0.1, and also changed the load balancing > port, and restarted. As a result, all other nodes in the cluster stopped > sending data to the node and started generating ConnectExceptions. The other > nodes in the cluster should still be able send after a load balance > hostname/port is changed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] exceptionfactory commented on pull request #5262: NIFI-8766 Implemented Public Key Algorithm for JWT Signing
exceptionfactory commented on pull request #5262: URL: https://github.com/apache/nifi/pull/5262#issuecomment-893981880 Following additional review of [RFC 7518 Section 8](https://datatracker.ietf.org/doc/html/rfc7518#section-8.3), I adjusted the signing algorithm to `PS512` instead of `RS512` in order to use `RSASSA-PSS` instead of `RSASSA-PKCS1-v1_5`. The Nimbus JWT library supports both algorithms and as a self-contained implementation, this provides better security for signing with the same RSA Key Pair approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5262: NIFI-8766 Implemented RS512 Algorithm for JWT Signing
exceptionfactory commented on a change in pull request #5262:
URL: https://github.com/apache/nifi/pull/5262#discussion_r683928100
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/revocation/JwtRevocationService.java
##
@@ -14,36 +14,32 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.admin.service;
+package org.apache.nifi.web.security.jwt.revocation;
-import org.apache.nifi.key.Key;
+import java.time.Instant;
/**
- * Manages NiFi user keys.
+ * JSON Web Token Revocation Service
*/
-public interface KeyService {
-
+public interface JwtRevocationService {
/**
- * Gets a key for the specified user identity. Returns null if the user
has not had a key issued
- *
- * @param id The key id
- * @return The key or null
+ * Delete Expired Revocations
*/
-Key getKey(int id);
+void deleteExpired();
/**
- * Gets a key for the specified user identity. If a key does not exist,
one will be created.
+ * Is JSON Web Token Identifier Revoked
*
- * @param identity The user identity
- * @return The key
- * @throws AdministrationException if it failed to get/create the key
+ * @param id JSON Web Token Identifier
+ * @return Revoked Status
*/
-Key getOrCreateKey(String identity);
+boolean isRevoked(String id);
/**
- * Deletes keys for the specified identity.
+ * Set Revoked Status using JSON Web Token Identifier
*
- * @param keyId The user's key ID
+ * @param id JSON Web Token Identifier
+ * @param expiration Expiration of Revocation Status
Review comment:
Thanks for the suggestion, I have updated the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5262: NIFI-8766 Implemented RS512 Algorithm for JWT Signing
exceptionfactory commented on a change in pull request #5262: URL: https://github.com/apache/nifi/pull/5262#discussion_r683926529 ## File path: nifi-docs/src/main/asciidoc/administration-guide.adoc ## @@ -489,6 +489,28 @@ To enable authentication via Apache Knox the following properties must be config this listing. The audience that is populated in the token can be configured in Knox. |== +[[json_web_token]] +=== JSON Web Tokens + +NiFi uses JSON Web Tokens to provide authenticated access after the initial login process. Generated JSON Web Tokens include the authenticated user identity +as well as the issuer and expiration from the configured Login Identity Provider. + +NiFi uses generated RSA Key Pairs with a key size of 4096 bits to support the `RS512` algorithm for JSON Web Signatures. The system stores RSA Review comment: Thanks for the follow up and background reference, that is helpful to know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] yoshiata commented on pull request #4943: NIFI-5886: Add RPM profile for NiFi Toolkit
yoshiata commented on pull request #4943: URL: https://github.com/apache/nifi/pull/4943#issuecomment-893958220 @pvillard31 @kotarot Thank you for reviewing and marging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-9013) Eliminate unnecessary dependency from nifi-data-provenance-utils on nifi-properties & loader
[ https://issues.apache.org/jira/browse/NIFI-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen resolved NIFI-9013. Fix Version/s: 1.15.0 Resolution: Fixed > Eliminate unnecessary dependency from nifi-data-provenance-utils on > nifi-properties & loader > > > Key: NIFI-9013 > URL: https://issues.apache.org/jira/browse/NIFI-9013 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework, NiFi Stateless >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.15.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The nifi-data-provenance-utils has a maven dependency on nifi-properties and > nifi-properties-loader but does not use them. Perhaps this was due to a > refactoring. > This dependency, though, adds about 10 MB of extra dependencies to > nifi-stateless, which means that it bloats the nifi-stateless assembly by > 25%. We should be able to simply remove the maven dependencies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #5283: NIFI-9013: Removed unnecessary dependencies
asfgit closed pull request #5283: URL: https://github.com/apache/nifi/pull/5283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9013) Eliminate unnecessary dependency from nifi-data-provenance-utils on nifi-properties & loader
[ https://issues.apache.org/jira/browse/NIFI-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394405#comment-17394405 ] ASF subversion and git services commented on NIFI-9013: --- Commit 787f16da3fe6c6660dd4c3d103432496e217bcf6 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=787f16d ] NIFI-9013: Moved nifi-properties-loader and nifi-properties dependencies from data-provenance-utils to the repository where they are needed This closes #5283 Signed-off-by: Mike Thomsen > Eliminate unnecessary dependency from nifi-data-provenance-utils on > nifi-properties & loader > > > Key: NIFI-9013 > URL: https://issues.apache.org/jira/browse/NIFI-9013 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework, NiFi Stateless >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The nifi-data-provenance-utils has a maven dependency on nifi-properties and > nifi-properties-loader but does not use them. Perhaps this was due to a > refactoring. > This dependency, though, adds about 10 MB of extra dependencies to > nifi-stateless, which means that it bloats the nifi-stateless assembly by > 25%. We should be able to simply remove the maven dependencies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] greyp9 commented on a change in pull request #5262: NIFI-8766 Implemented RS512 Algorithm for JWT Signing
greyp9 commented on a change in pull request #5262:
URL: https://github.com/apache/nifi/pull/5262#discussion_r683866639
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/revocation/JwtRevocationService.java
##
@@ -14,36 +14,32 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.admin.service;
+package org.apache.nifi.web.security.jwt.revocation;
-import org.apache.nifi.key.Key;
+import java.time.Instant;
/**
- * Manages NiFi user keys.
+ * JSON Web Token Revocation Service
*/
-public interface KeyService {
-
+public interface JwtRevocationService {
/**
- * Gets a key for the specified user identity. Returns null if the user
has not had a key issued
- *
- * @param id The key id
- * @return The key or null
+ * Delete Expired Revocations
*/
-Key getKey(int id);
+void deleteExpired();
/**
- * Gets a key for the specified user identity. If a key does not exist,
one will be created.
+ * Is JSON Web Token Identifier Revoked
*
- * @param identity The user identity
- * @return The key
- * @throws AdministrationException if it failed to get/create the key
+ * @param id JSON Web Token Identifier
+ * @return Revoked Status
*/
-Key getOrCreateKey(String identity);
+boolean isRevoked(String id);
/**
- * Deletes keys for the specified identity.
+ * Set Revoked Status using JSON Web Token Identifier
*
- * @param keyId The user's key ID
+ * @param id JSON Web Token Identifier
+ * @param expiration Expiration of Revocation Status
Review comment:
Got it; thanks. So maybe the javadoc comment could reflect that
somehow? Suggestion:
> retention period for revocation status record
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a change in pull request #5262: NIFI-8766 Implemented RS512 Algorithm for JWT Signing
greyp9 commented on a change in pull request #5262: URL: https://github.com/apache/nifi/pull/5262#discussion_r683865470 ## File path: nifi-docs/src/main/asciidoc/administration-guide.adoc ## @@ -489,6 +489,28 @@ To enable authentication via Apache Knox the following properties must be config this listing. The audience that is populated in the token can be configured in Knox. |== +[[json_web_token]] +=== JSON Web Tokens + +NiFi uses JSON Web Tokens to provide authenticated access after the initial login process. Generated JSON Web Tokens include the authenticated user identity +as well as the issuer and expiration from the configured Login Identity Provider. + +NiFi uses generated RSA Key Pairs with a key size of 4096 bits to support the `RS512` algorithm for JSON Web Signatures. The system stores RSA Review comment: I see the recent industry movement to 3072-bit minimum RSA key lengths. 4096 is probably a good choice to future proof this implementation. I have concerns about performance of key generation at startup on lower-end devices, but this is probably not a valid concern for NiFi. https://knowledge.digicert.com/alerts/code-signing-new-minimum-rsa-keysize.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9016) Add BCFKS KeyStoreKeyProvider Example to User Guide
[
https://issues.apache.org/jira/browse/NIFI-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-9016:
---
Labels: security (was: )
Status: Patch Available (was: Open)
> Add BCFKS KeyStoreKeyProvider Example to User Guide
> ---
>
> Key: NIFI-9016
> URL: https://issues.apache.org/jira/browse/NIFI-9016
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The NiFi User Guide currently includes documentation on the
> {{KeyStoreKeyProvider}} with an example command for PKCS12 Secret Key
> generation and configuration. Using {{keytool}} to generate a Secret Key for
> storage in BCFKS requires additional arguments and reference to a separate
> provider JAR. Adding an example to the documentation would make it easier to
> configure repository encryption using BCFKS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] exceptionfactory opened a new pull request #5285: NIFI-9016 Added BCFKS KeyStoreKeyProvider examples to User Guide
exceptionfactory opened a new pull request #5285:
URL: https://github.com/apache/nifi/pull/5285
Description of PR
NIFI-9016 Updates the User Guide for encrypted repositories to include
examples for generating and storing AES Secret Keys using the BouncyCastle FIPS
KeyStore (BCFKS) format. Changes include consolidating duplicate examples into
a single section, and providing links from each `KeyStoreKeyProvider` encrypted
repository section to the new examples.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [X] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8990) Upgrade Groovy to 2.5.14 and 2.4.21
[ https://issues.apache.org/jira/browse/NIFI-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394314#comment-17394314 ] ASF subversion and git services commented on NIFI-8990: --- Commit 47d5ae5d75ee94b4f8613f7eccfced243d2a4d89 in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=47d5ae5 ] NIFI-8990: Downgrade Gremlin from 3.5.1 to 3.4.4 to support Graph DBs This closes #5284 Signed-off-by: David Handermann > Upgrade Groovy to 2.5.14 and 2.4.21 > --- > > Key: NIFI-8990 > URL: https://issues.apache.org/jira/browse/NIFI-8990 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry, Tools and > Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: security > Fix For: 1.15.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Groovy versions 2.5.13 and 2.4.20 and have a vulnerability related to > directory creation in extension methods as described in > [CVE-2020-17521|https://nvd.nist.gov/vuln/detail/CVE-2020-17521]. Groovy > 2.5.14 and 2.4.21 address this problem as well as other vulnerabilities in > earlier versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8990) Upgrade Groovy to 2.5.14 and 2.4.21
[ https://issues.apache.org/jira/browse/NIFI-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8990: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Groovy to 2.5.14 and 2.4.21 > --- > > Key: NIFI-8990 > URL: https://issues.apache.org/jira/browse/NIFI-8990 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry, Tools and > Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: security > Fix For: 1.15.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Groovy versions 2.5.13 and 2.4.20 and have a vulnerability related to > directory creation in extension methods as described in > [CVE-2020-17521|https://nvd.nist.gov/vuln/detail/CVE-2020-17521]. Groovy > 2.5.14 and 2.4.21 address this problem as well as other vulnerabilities in > earlier versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #5284: NIFI-8990: Downgrade Gremlin from 3.5.1 to 3.4.4 to support Graph DBs
asfgit closed pull request #5284: URL: https://github.com/apache/nifi/pull/5284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-9017) If a node's load balance host/port is updated, other nodes stop sending data
Mark Payne created NIFI-9017: Summary: If a node's load balance host/port is updated, other nodes stop sending data Key: NIFI-9017 URL: https://issues.apache.org/jira/browse/NIFI-9017 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne I have a 3-node cluster. On one of the nodes, I stopped it, changed the load balance host from localhost to 127.0.0.1, and also changed the load balancing port, and restarted. As a result, all other nodes in the cluster stopped sending data to the node and started generating ConnectExceptions. The other nodes in the cluster should still be able send after a load balance hostname/port is changed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-9016) Add BCFKS KeyStoreKeyProvider Example to User Guide
David Handermann created NIFI-9016:
--
Summary: Add BCFKS KeyStoreKeyProvider Example to User Guide
Key: NIFI-9016
URL: https://issues.apache.org/jira/browse/NIFI-9016
Project: Apache NiFi
Issue Type: Improvement
Components: Documentation Website
Reporter: David Handermann
Assignee: David Handermann
The NiFi User Guide currently includes documentation on the
{{KeyStoreKeyProvider}} with an example command for PKCS12 Secret Key
generation and configuration. Using {{keytool}} to generate a Secret Key for
storage in BCFKS requires additional arguments and reference to a separate
provider JAR. Adding an example to the documentation would make it easier to
configure repository encryption using BCFKS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8990) Upgrade Groovy to 2.5.14 and 2.4.21
[ https://issues.apache.org/jira/browse/NIFI-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8990: --- Affects Version/s: (was: 1.15.0) (was: 1.14.0) Status: Patch Available (was: Reopened) > Upgrade Groovy to 2.5.14 and 2.4.21 > --- > > Key: NIFI-8990 > URL: https://issues.apache.org/jira/browse/NIFI-8990 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry, Tools and > Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: security > Fix For: 1.15.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Groovy versions 2.5.13 and 2.4.20 and have a vulnerability related to > directory creation in extension methods as described in > [CVE-2020-17521|https://nvd.nist.gov/vuln/detail/CVE-2020-17521]. Groovy > 2.5.14 and 2.4.21 address this problem as well as other vulnerabilities in > earlier versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8990) Upgrade Groovy to 2.5.14 and 2.4.21
[ https://issues.apache.org/jira/browse/NIFI-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8990: --- Status: Reopened (was: Reopened) > Upgrade Groovy to 2.5.14 and 2.4.21 > --- > > Key: NIFI-8990 > URL: https://issues.apache.org/jira/browse/NIFI-8990 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry, Tools and > Build >Affects Versions: 1.14.0, 1.15.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: security > Fix For: 1.15.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Groovy versions 2.5.13 and 2.4.20 and have a vulnerability related to > directory creation in extension methods as described in > [CVE-2020-17521|https://nvd.nist.gov/vuln/detail/CVE-2020-17521]. Groovy > 2.5.14 and 2.4.21 address this problem as well as other vulnerabilities in > earlier versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 opened a new pull request #5284: NIFI-8990: Downgrade Gremlin from 3.5.1 to 3.4.4 to support Graph DBs
mattyb149 opened a new pull request #5284:
URL: https://github.com/apache/nifi/pull/5284
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Changes the Gremlin version to 3.4.4 from 3.5.1 while retaining the Groovy
upgrade to 2.5.14
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [x] Have you verified that the full build is successful on JDK 11?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Reopened] (NIFI-8990) Upgrade Groovy to 2.5.14 and 2.4.21
[ https://issues.apache.org/jira/browse/NIFI-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reopened NIFI-8990: Reopening to downgrade Gremlin to 3.4.4 to support current graph database implementations. The Groovy upgrade should remain however. > Upgrade Groovy to 2.5.14 and 2.4.21 > --- > > Key: NIFI-8990 > URL: https://issues.apache.org/jira/browse/NIFI-8990 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry, Tools and > Build >Affects Versions: 1.14.0, 1.15.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: security > Fix For: 1.15.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Groovy versions 2.5.13 and 2.4.20 and have a vulnerability related to > directory creation in extension methods as described in > [CVE-2020-17521|https://nvd.nist.gov/vuln/detail/CVE-2020-17521]. Groovy > 2.5.14 and 2.4.21 address this problem as well as other vulnerabilities in > earlier versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-9015) Ability to derive custom Rest API processes from InvokeHTTP
[ https://issues.apache.org/jira/browse/NIFI-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler updated NIFI-9015: -- Description: In setups with custom development, there is a lot of boilerplate customization to InvokeHTTP around some known set of rest apis. As such flow developers / users have to 'know' and understand these things in order to setup possibly multiple InvokeHTTP instances with these details. Some users may instead want to create a top level derived processor with custom setup and parameters for a specific rest api for easy configuration and deployment. The [InvokeAWSGatewayApi|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.14.0/org.apache.nifi.processors.aws.wag.InvokeAWSGatewayApi/index.html] processor ( which itself had to be copied from InvokeHTTP because of this limitation ) offers this, such that you can create a derived processor for a custom AWS Web Gateway service. With a coming processor registry, the ability for people to contribute specialized processors for certain APIs would be great. These may be a subset or a super set of the existing properties. We may also move the 'base' properties to a new UI tab in the configuration, such that you can add custom extra configuration in the first focus tab. We could also allow for removing properties as well from the base tab, etc etc This task would involve refactoring the invokeHTTP processor such that there is a reusable base, with the InvokeHTTP processor being a pass-through default implementation ( IE> a new derived from base would have all the same features as InvokeHTTP ). was: In setups with custom development, there is a lot of boilerplate customization to InvokeHTTP around some known set of rest apis. As such flow developers / users have to 'know' and understand these things in order to setup possibly multiple InvokeHTTP instances with these details. Some users may instead want to create a top level derived processor with custom setup and parameters for a specific rest api for easy configuration and deployment. The [InvokeAWSGatewayApi|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.14.0/org.apache.nifi.processors.aws.wag.InvokeAWSGatewayApi/index.html] processor ( which itself had to be copied from InvokeHTTP because of this limitation ) offers this, such that you can create a derived processor for a custom AWS Web Gateway service. With a coming processor registry, the ability for people to contribute specialized processors for certain APIs would be great. This task would involve refactoring the invokeHTTP processor such that there is a reusable base, with the InvokeHTTP processor being a pass-through default implementation ( IE> a new derived from base would have all the same features as InvokeHTTP ). > Ability to derive custom Rest API processes from InvokeHTTP > --- > > Key: NIFI-9015 > URL: https://issues.apache.org/jira/browse/NIFI-9015 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Otto Fowler >Priority: Major > > In setups with custom development, there is a lot of boilerplate > customization to InvokeHTTP around some known set of rest apis. As such flow > developers / users have to 'know' and understand these things in order to > setup possibly multiple InvokeHTTP instances with these details. > Some users may instead want to create a top level derived processor with > custom setup and parameters for a specific rest api for easy configuration > and deployment. > The > [InvokeAWSGatewayApi|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.14.0/org.apache.nifi.processors.aws.wag.InvokeAWSGatewayApi/index.html] > processor ( which itself had to be copied from InvokeHTTP because of this > limitation ) offers this, such that you can create a derived processor for a > custom AWS Web Gateway service. > With a coming processor registry, the ability for people to contribute > specialized processors for certain APIs would be great. These may be a > subset or a super set of the existing properties. We may also move the > 'base' properties to a new UI tab in the configuration, such that you can add > custom extra configuration in the first focus tab. We could also allow for > removing properties as well from the base tab, etc etc > This task would involve refactoring the invokeHTTP processor such that there > is a reusable base, with the InvokeHTTP processor being a pass-through > default implementation ( IE> a new derived from base would have all the same > features as InvokeHTTP ). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-9015) Ability to derive custom Rest API processes from InvokeHTTP
Otto Fowler created NIFI-9015: - Summary: Ability to derive custom Rest API processes from InvokeHTTP Key: NIFI-9015 URL: https://issues.apache.org/jira/browse/NIFI-9015 Project: Apache NiFi Issue Type: Improvement Reporter: Otto Fowler In setups with custom development, there is a lot of boilerplate customization to InvokeHTTP around some known set of rest apis. As such flow developers / users have to 'know' and understand these things in order to setup possibly multiple InvokeHTTP instances with these details. Some users may instead want to create a top level derived processor with custom setup and parameters for a specific rest api for easy configuration and deployment. The [InvokeAWSGatewayApi|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.14.0/org.apache.nifi.processors.aws.wag.InvokeAWSGatewayApi/index.html] processor ( which itself had to be copied from InvokeHTTP because of this limitation ) offers this, such that you can create a derived processor for a custom AWS Web Gateway service. With a coming processor registry, the ability for people to contribute specialized processors for certain APIs would be great. This task would involve refactoring the invokeHTTP processor such that there is a reusable base, with the InvokeHTTP processor being a pass-through default implementation ( IE> a new derived from base would have all the same features as InvokeHTTP ). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-9014) Elastic JDBC connection hanging
Heinz Mayer created NIFI-9014: - Summary: Elastic JDBC connection hanging Key: NIFI-9014 URL: https://issues.apache.org/jira/browse/NIFI-9014 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.14.0 Reporter: Heinz Mayer After the upgrade from 1.11.4 to 1.14.0 the elasticsearch JDBC connection doesn't work. ExecuteSQL is hanging and doesn't log any error message - the same flow works without a problem in 1.11.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1567) Enable linter check in all extensions
[ https://issues.apache.org/jira/browse/MINIFICPP-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-1567. Resolution: Fixed [https://github.com/apache/nifi-minifi-cpp/pull/1103] [https://github.com/apache/nifi-minifi-cpp/pull/1112] [https://github.com/apache/nifi-minifi-cpp/pull/1097] [https://github.com/apache/nifi-minifi-cpp/pull/1089] The linter is now enabled on every extension except tensorflow (which has build issues atm (not on ci)) The tensorflow extension linter check should be enabled when the build issues are resolved. > Enable linter check in all extensions > - > > Key: MINIFICPP-1567 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1567 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Time Spent: 5h 40m > Remaining Estimate: 0h > > Currently only libminifi/src,libminifi/include,libminifi/target and > extensions with the register_extension_linter macro are checked with the > linter. > We should add this macro to every extension and fix related linter issues > - ✅ aws > - ✅ azure > - ✅ bustache [-PR#1089-|https://github.com/apache/nifi-minifi-cpp/pull/1089] > - ✅ civetweb > [-PR#1089-|https://github.com/apache/nifi-minifi-cpp/pull/1089] > - ✅ coap [-PR#1089-|https://github.com/apache/nifi-minifi-cpp/pull/1089] > - ✅ expression-language > [-PR#1097-|https://github.com/apache/nifi-minifi-cpp/pull/1097] > - ✅ gps [-PR#1097-|https://github.com/apache/nifi-minifi-cpp/pull/1097] > - ✅ http-curl [-PR#1097-|https://github.com/apache/nifi-minifi-cpp/pull/1097] > - ✅ jni [-PR#1112-|https://github.com/apache/nifi-minifi-cpp/pull/1112] > - ✅ libarchive > [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ librdkafka > - ✅ mqtt [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ opc [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ opencv [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ openwsman [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ pcap [-PR#1103-|https://github.com/apache/nifi-minifi-cpp/pull/1103] > - ✅ pdh > - ✅ rockdb-repos > -[PR#1112|https://github.com/apache/nifi-minifi-cpp/pull/1112]- > - ✅ script [-PR#1112-|https://github.com/apache/nifi-minifi-cpp/pull/1112] > - ✅ sensors [-PR#1112-|https://github.com/apache/nifi-minifi-cpp/pull/1112] > - ✅ sftp [-PR#1112-|https://github.com/apache/nifi-minifi-cpp/pull/1112] > - ✅ sql > - ✅ standard-processors > - ❌ tensorflow > - ✅ usb-camera > [-PR#1097-|https://github.com/apache/nifi-minifi-cpp/pull/1097] > - ✅ windows-event-log > [-PR#1089-|https://github.com/apache/nifi-minifi-cpp/pull/1089] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1593) Create linux equivalent to PerformanceDataMonitor processor
[ https://issues.apache.org/jira/browse/MINIFICPP-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1593: --- Status: Patch Available (was: In Progress) [https://github.com/apache/nifi-minifi-cpp/pull/1152] > Create linux equivalent to PerformanceDataMonitor processor > > > Key: MINIFICPP-1593 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1593 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Create a similar processor to the PerformanceDataMonitor processor on linux > (maybe macOS?) > It should be able to monitor > +CPU+: > * Idle (%) > * Stolen > * System > * User > +Memory:+ > * Total > * Used (%) > * Free (%) > * Swap Free > * Swap Used > +I/O+: > * Read > * Write > * Avg Utilization > +Disk+: > * Total > * Free > * Used > * Volumes/Partitions > +Network+: > * Avg Utilization (%) > * In Errors > * Out Errors > +Process+ > * Process ID > * Memory > * Processor Time -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1573) Make AppendHostInfo platform independent
[ https://issues.apache.org/jira/browse/MINIFICPP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1573: --- Resolution: Fixed Status: Resolved (was: Patch Available) https://github.com/apache/nifi-minifi-cpp/commit/18e79b65ac25b849fa4937159546f41d4c08d568 > Make AppendHostInfo platform independent > > > Key: MINIFICPP-1573 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1573 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 4h 50m > Remaining Estimate: 0h > > Currently AppendHostInfo only appends hostname on windows, the ip address > appending is only a linux/mac feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1617) Update CONTRIB.md where it differs from practice
[
https://issues.apache.org/jira/browse/MINIFICPP-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1617:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Update CONTRIB.md where it differs from practice
>
>
> Key: MINIFICPP-1617
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1617
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.11.0
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> CONTRIB.md refers to the google style guide, but the codebase and practice
> differs from that in many ways.
> A quick list:
> * we use .cpp, not .cc for impl files
> * New code should prefer\{{ #pragma once}} over include guards.
> * I think forward declarations are fine in the codebase.
> * I think using-directives ({{using namespace}}) are acceptable, but not
> encouraged in our codebase.
> * We use objects with static storage duration in quite a few places, e.g.
> processor properties. Most of these don't follow the rule of being trivially
> destructible and initialized with a constant expression.
> * We use inheritance way more than is recommended by the style guide. I
> think we should improve this, so will not include as an exception.
> * We're more liberal regarding operator overloading and user-defined
> literals.
> * We allow public mutable data members
> * We allow functions to be defined inline when it makes sense
> * We have default arguments on virtual functions in many cases. We should
> improve on this, so I'm not including it in CONTRIB.md.
> * We use rvalue references
> * We use exceptions
> * We use RTTI
> * We use {{gsl::narrow}} and {{gsl::narrow_cast}} in addition to standard
> casts. We don't use abseil.
> * We're more liberal regarding the use of {{auto}}. The style guide says use
> it only when it makes the code clearer, but I think we allow using it
> according to personal preferences.
> * I think we're OK with template metaprogramming as long as the usage is
> clear.
> * In naming, we use lowerCamelCase for functions, including
> accessors/mutators and usually UPPER_SNAKE_CASE for constants. For filenames,
> we usually use the class name or some reasonable name typically in
> UpperCamelCase, except when it's meant to imitate some other header, like
> gsl.h. If a type name is imitating something from STL, boost or similar, then
> STL-style lower_snake_case is used (e.g. {{utils::make_unique}}).
> * Static data members might be named differently than
> snake_with_trailing_underscore_, I can't recall a good example atm. I think
> they are snake_case.
> * Enums are either FULL_CAPS_SNAKE or UpperCamelCase.
> * We don't use file-level comments to describe the contents of a file, we
> only include the license boilerplate.
> * Function comments use Javadoc style {{/** */}}
> * Line length is not really limited, but the linter checks for 200
> characters. Use NOLINT comment when it makes sense to have a longer line.
> * continuation indentaion is ok both with 2 levels and aligned
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-9012) InvokeHttps
[
https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393951#comment-17393951
]
Michal Šunka edited comment on NIFI-9012 at 8/5/21, 3:16 PM:
-
[~pvillard] - Are you interested in anything specific in log files? To be
honest I am not wiling to share those not-anonymized and doing so would take
quite some time. But in the logs for 80 minutes prior first restart and up to
the third restart no error stacktraces are present (aside from those concerning
unavailable site-to-site server during restart).
Here are anonymized [^nifi.properties].
Concerning repository:
{{
/opt/nifi/nifi-current$ ls -l | grep repo
drwxr-xr-x. 1026 nifi nifi 20480 Aug 4 12:58 content_repository
drwxr-xr-x. 2 nifi nifi 100 Aug 4 12:58 database_repository
drwxr-xr-x. 4 nifi nifi 52 Aug 5 11:57 flowfile_repository
drwxr-xr-x. 4 nifi nifi 8192 Aug 5 11:55 provenance_repository
}}
But it is from now when all is OK. Or you had something different in mind?
And container as a whole has no disk limit I am aware of. My other instance
(with similar settings, version 1.12.1) had to made 10 GB of logs to stop
working due to the no-space errors. Some container settings:
[^docker_inspect_excerpt.txt]
was (Author: raocz):
[~pvillard] - Are you interested in anything specific in log files? To be
honest I am not wiling to share those not-anonymized and doing so would take
quite some time. But in the logs for 80 minutes prior first restart and up to
the third restart no error stacktraces are present (aside from those concerning
unavailable site-to-site server during restart).
Here are anonymized [^nifi.properties].
{{Concerning repository:
/opt/nifi/nifi-current$ ls -l | grep repo
drwxr-xr-x. 1026 nifi nifi 20480 Aug 4 12:58 content_repository
drwxr-xr-x. 2 nifi nifi 100 Aug 4 12:58 database_repository
drwxr-xr-x. 4 nifi nifi 52 Aug 5 11:57 flowfile_repository
drwxr-xr-x. 4 nifi nifi 8192 Aug 5 11:55 provenance_repository
}}
But it is from now when all is OK. Or you had something different in mind?
And container as a whole has no disk limit I am aware of. My other instance
(with similar settings, version 1.12.1) had to made 10 GB of logs to stop
working due to the no-space errors. Some container settings:
[^docker_inspect_excerpt.txt]
> InvokeHttps
> ---
>
> Key: NIFI-9012
> URL: https://issues.apache.org/jira/browse/NIFI-9012
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.14.0
> Environment: Deckerized NiFi with container from Docker Hub, no
> custom processors
>Reporter: Michal Šunka
>Priority: Major
> Attachments: docker_inspect_excerpt.txt, nifi.properties,
> thread-dump.txt, thread-dump2.txt
>
>
> I was running a NiFi version 1.12.1 without any problems with this regard.
> Yesterday I updated to version 1.14.0 (really wanted to get fix
> https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse
> state.
> I use InvokeHTTP processor to self-connect to NiFi API to get count of
> flowfiles being processed by given processor group.
> However this InvokeHTTP started getting stuck. Processor would just spin up a
> new thread and this will sit there indefinitely (six hours at-least),
> blocking incoming flowfile. After forceful termination of processor (simple
> stop is not enough) the flowfile would get released and next start will
> consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10
> seconds and idle 1 minute.) And of course log is clear of any errors which
> seems to be connected to this...
> I had noticed similar issue also with other processors, namely MergeContent
> processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes
> (merge bin max age is 5 seconds). After forced termination all starts working
> (for some time - a few dozens of minutes).
> Oh and in all cases the thread hangs in terminating state (red thread icon
> with a digit in parenthesis; the digit sits there all the time and wont
> disappear). Only restart of whole NiFi clears the issue. For some time.
>
> Maybe some disk I/O issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-9012) InvokeHttps
[
https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393993#comment-17393993
]
Michal Šunka edited comment on NIFI-9012 at 8/5/21, 3:15 PM:
-
I found those somewhat close-sounding lines:
{code:java}
2021-08-05 09:54:08,150 INFO [pool-10-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2021-08-05 09:54:08,151 INFO [pool-10-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 46 records in 0 milliseconds
2021-08-05 09:54:11,271 INFO [Cleanup Archive for default]
o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes)
from archive
2021-08-05 09:54:11,271 INFO [Timer-Driven Process Thread-1]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,368 INFO [Timer-Driven Process Thread-2]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,374 INFO [Timer-Driven Process Thread-6]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-4]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-9 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-3 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-7 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-8 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-11 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-13 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-17]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-12 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-5 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-18 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-15 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-19]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-16 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-20 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-14 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-10 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process
[jira] [Created] (NIFI-9013) Eliminate unnecessary dependency from nifi-data-provenance-utils on nifi-properties & loader
Mark Payne created NIFI-9013: Summary: Eliminate unnecessary dependency from nifi-data-provenance-utils on nifi-properties & loader Key: NIFI-9013 URL: https://issues.apache.org/jira/browse/NIFI-9013 Project: Apache NiFi Issue Type: Task Components: Core Framework, NiFi Stateless Reporter: Mark Payne Assignee: Mark Payne The nifi-data-provenance-utils has a maven dependency on nifi-properties and nifi-properties-loader but does not use them. Perhaps this was due to a refactoring. This dependency, though, adds about 10 MB of extra dependencies to nifi-stateless, which means that it bloats the nifi-stateless assembly by 25%. We should be able to simply remove the maven dependencies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #5283: NIFI-9013: Removed unnecessary dependencies
markap14 opened a new pull request #5283:
URL: https://github.com/apache/nifi/pull/5283
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on pull request #1147: MINIFICPP-1289 add range-v3 and examples
fgerlits commented on pull request #1147: URL: https://github.com/apache/nifi-minifi-cpp/pull/1147#issuecomment-893497968 Using an unreleased version until there is a 0.11.1 release available is OK. I'd probably use the newest version available instead of the oldest version which contains this fix, but I'm fine with either option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8753) Expose flowfile repository implementation configuration
[
https://issues.apache.org/jira/browse/NIFI-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-8753:
---
Status: Patch Available (was: Open)
> Expose flowfile repository implementation configuration
> ---
>
> Key: NIFI-8753
> URL: https://issues.apache.org/jira/browse/NIFI-8753
> Project: Apache NiFi
> Issue Type: Improvement
> Components: MiNiFi
>Reporter: Mauro Riva
>Priority: Minor
> Original Estimate: 1h
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The actual code sets the setting:
> {code:java}
> nifi.flowfile.repository.implementation{code}
> to the default value:
> {code:java}
> org.apache.nifi.controller.repository.WriteAheadFlowFileRepository{code}
> This setting can be exposed to modify it (e.g.
> org.apache.nifi.controller.repository.VolatileFlowFileRepository) using the
> config.yml file.
>
> More info about the flowfile settings
> [here|https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.0/nifi-system-properties/content/flowfile-repository.html].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1157: MINIFICPP-1624 Allow building in docker without up-to-date compilers …
szaszm commented on a change in pull request #1157:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1157#discussion_r683464208
##
File path: CMakeLists.txt
##
@@ -428,23 +460,17 @@ if (ENABLE_ALL OR ENABLE_COAP STREQUAL "ON")
endif()
endif()
-if (WIN32)
- option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
- if (ENABLE_ALL OR ENABLE_WEL)
-
- include(BundledPugiXml)
- use_bundled_pugixml(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_BINARY_DIR})
- createExtension(WEL-EXTENSION "WEL EXTENSIONS" "Enables the
suite of Windows Event Log extensions." "extensions/windows-event-log"
"extensions/windows-event-log/tests")
- endif()
-endif(WIN32)
+if (WIN32 AND (ENABLE_ALL OR ENABLE_WEL))
+ include(BundledPugiXml)
+ use_bundled_pugixml(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_BINARY_DIR})
+ createExtension(WEL-EXTENSION "WEL EXTENSIONS" "Enables the suite of
Windows Event Log extensions." "extensions/windows-event-log"
"extensions/windows-event-log/tests")
+endif(WIN32 AND (ENABLE_ALL OR ENABLE_WEL))
Review comment:
Changed in eaf13e4
##
File path: CMakeLists.txt
##
@@ -96,6 +83,59 @@ set_property(CACHE STRICT_GSL_CHECKS PROPERTY STRINGS
${STRICT_GSL_CHECKS_Values
if (WIN32)
option(MSI_REDISTRIBUTE_UCRT_NONASL "Redistribute Universal C Runtime
DLLs with the MSI generated by CPack. The resulting MSI is not distributable
under Apache 2.0." OFF)
+ option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
+ option(ENABLE_PDH "Enables PDH support." OFF)
+endif()
+
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+ option(ENABLE_SYSTEMD "Disables the systemd extension." ON)
+endif()
+
+option(DISABLE_EXPRESSION_LANGUAGE "Disables the scripting extensions." OFF)
+option(DISABLE_CIVET "Disables CivetWeb components." OFF)
+option(DISABLE_ROCKSDB "Disables the RocksDB extension." OFF)
+option(DISABLE_LIBARCHIVE "Disables the lib archive extensions." OFF)
+option(DISABLE_LZMA "Disables the liblzma build" OFF)
+option(DISABLE_BZIP2 "Disables the bzip2 build" OFF)
+option(ENABLE_GPS "Enables the GPS extension." OFF)
+option(ENABLE_COAP "Enables the CoAP extension." OFF)
+option(ENABLE_SQL "Enables the SQL Suite of Tools." OFF)
+option(ENABLE_MQTT "Enables the mqtt extension." OFF)
+option(ENABLE_PCAP "Enables the PCAP extension." OFF)
+option(ENABLE_LIBRDKAFKA "Enables the librdkafka extension." OFF)
+option(DISABLE_SCRIPTING "Disables the scripting extensions." OFF)
+option(ENABLE_SENSORS "Enables the Sensors package." OFF)
+option(ENABLE_USB_CAMERA "Enables USB camera support." OFF)
+option(ENABLE_TENSORFLOW "Disables the TensorFlow extensions." OFF) ##
Disabled by default because TF can be complex/environment-specific to build
Review comment:
fixed in eaf13e4
##
File path: CMakeLists.txt
##
@@ -96,6 +83,59 @@ set_property(CACHE STRICT_GSL_CHECKS PROPERTY STRINGS
${STRICT_GSL_CHECKS_Values
if (WIN32)
option(MSI_REDISTRIBUTE_UCRT_NONASL "Redistribute Universal C Runtime
DLLs with the MSI generated by CPack. The resulting MSI is not distributable
under Apache 2.0." OFF)
+ option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
+ option(ENABLE_PDH "Enables PDH support." OFF)
+endif()
+
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+ option(ENABLE_SYSTEMD "Disables the systemd extension." ON)
Review comment:
fixed in eaf13e4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1157: MINIFICPP-1624 Allow building in docker without up-to-date compilers …
fgerlits commented on a change in pull request #1157:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1157#discussion_r683444095
##
File path: CMakeLists.txt
##
@@ -96,6 +83,59 @@ set_property(CACHE STRICT_GSL_CHECKS PROPERTY STRINGS
${STRICT_GSL_CHECKS_Values
if (WIN32)
option(MSI_REDISTRIBUTE_UCRT_NONASL "Redistribute Universal C Runtime
DLLs with the MSI generated by CPack. The resulting MSI is not distributable
under Apache 2.0." OFF)
+ option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
+ option(ENABLE_PDH "Enables PDH support." OFF)
+endif()
+
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+ option(ENABLE_SYSTEMD "Disables the systemd extension." ON)
Review comment:
the description should say "Enables ..."
##
File path: CMakeLists.txt
##
@@ -428,23 +460,17 @@ if (ENABLE_ALL OR ENABLE_COAP STREQUAL "ON")
endif()
endif()
-if (WIN32)
- option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
- if (ENABLE_ALL OR ENABLE_WEL)
-
- include(BundledPugiXml)
- use_bundled_pugixml(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_BINARY_DIR})
- createExtension(WEL-EXTENSION "WEL EXTENSIONS" "Enables the
suite of Windows Event Log extensions." "extensions/windows-event-log"
"extensions/windows-event-log/tests")
- endif()
-endif(WIN32)
+if (WIN32 AND (ENABLE_ALL OR ENABLE_WEL))
+ include(BundledPugiXml)
+ use_bundled_pugixml(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_BINARY_DIR})
+ createExtension(WEL-EXTENSION "WEL EXTENSIONS" "Enables the suite of
Windows Event Log extensions." "extensions/windows-event-log"
"extensions/windows-event-log/tests")
+endif(WIN32 AND (ENABLE_ALL OR ENABLE_WEL))
Review comment:
I would change this to `endif()`
##
File path: CMakeLists.txt
##
@@ -96,6 +83,59 @@ set_property(CACHE STRICT_GSL_CHECKS PROPERTY STRINGS
${STRICT_GSL_CHECKS_Values
if (WIN32)
option(MSI_REDISTRIBUTE_UCRT_NONASL "Redistribute Universal C Runtime
DLLs with the MSI generated by CPack. The resulting MSI is not distributable
under Apache 2.0." OFF)
+ option(ENABLE_WEL "Enables the suite of Windows Event Log extensions."
OFF)
+ option(ENABLE_PDH "Enables PDH support." OFF)
+endif()
+
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+ option(ENABLE_SYSTEMD "Disables the systemd extension." ON)
+endif()
+
+option(DISABLE_EXPRESSION_LANGUAGE "Disables the scripting extensions." OFF)
+option(DISABLE_CIVET "Disables CivetWeb components." OFF)
+option(DISABLE_ROCKSDB "Disables the RocksDB extension." OFF)
+option(DISABLE_LIBARCHIVE "Disables the lib archive extensions." OFF)
+option(DISABLE_LZMA "Disables the liblzma build" OFF)
+option(DISABLE_BZIP2 "Disables the bzip2 build" OFF)
+option(ENABLE_GPS "Enables the GPS extension." OFF)
+option(ENABLE_COAP "Enables the CoAP extension." OFF)
+option(ENABLE_SQL "Enables the SQL Suite of Tools." OFF)
+option(ENABLE_MQTT "Enables the mqtt extension." OFF)
+option(ENABLE_PCAP "Enables the PCAP extension." OFF)
+option(ENABLE_LIBRDKAFKA "Enables the librdkafka extension." OFF)
+option(DISABLE_SCRIPTING "Disables the scripting extensions." OFF)
+option(ENABLE_SENSORS "Enables the Sensors package." OFF)
+option(ENABLE_USB_CAMERA "Enables USB camera support." OFF)
+option(ENABLE_TENSORFLOW "Disables the TensorFlow extensions." OFF) ##
Disabled by default because TF can be complex/environment-specific to build
Review comment:
old typo, but should be "Enables ..."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1624) Allow building in docker without up-to-date compilers on the host
[ https://issues.apache.org/jira/browse/MINIFICPP-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1624: Status: Patch Available (was: In Progress) https://github.com/apache/nifi-minifi-cpp/pull/1157 > Allow building in docker without up-to-date compilers on the host > - > > Key: MINIFICPP-1624 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1624 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > After MINIFICPP-1425, there is no reason to require C++20-compatible > compilers on hosts that are only used to launch docker containers for > building. Please note that cmake still needs _some_ compiler and generator > target (e.g. make or ninja) to be installed on the host to be able to > generate a build system. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-9012) InvokeHttps
[
https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393993#comment-17393993
]
Michal Šunka commented on NIFI-9012:
I found those somewhat close-sounding lines:
{code:java}
2021-08-05 09:54:08,150 INFO [pool-10-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2021-08-05 09:54:08,151 INFO [pool-10-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 46 records in 0 milliseconds
2021-08-05 09:54:11,271 INFO [Cleanup Archive for default]
o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes)
from archive
2021-08-05 09:54:11,271 INFO [Timer-Driven Process Thread-1]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,368 INFO [Timer-Driven Process Thread-2]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,374 INFO [Timer-Driven Process Thread-6]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-4]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-9 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-3 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-7 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-8 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-11 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-13 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-17]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-12 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-5 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-18 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,375 INFO [Timer-Driven Process Thread-15 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-19]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-16 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-20 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-14 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-10 ]
o.a.n.c.repository.FileSystemRepository Unable to write to container default
due to archive file size constraints; waiting for archive cleanup
2021-08-05 09:54:11,376 INFO [Timer-Driven Process Thread-23]
o.a.n.c.repository.FileSystemRepository
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request #1157: MINIFICPP-1624 Allow building in docker without up-to-date compilers …
szaszm opened a new pull request #1157: URL: https://github.com/apache/nifi-minifi-cpp/pull/1157 …on the host https://issues.apache.org/jira/browse/MINIFICPP-1624 After MINIFICPP-1425, there is no reason to require C++20-compatible compilers on hosts that are only used to launch docker containers for building. Please note that cmake still needs some compiler and generator target (e.g. make or ninja) to be installed on the host to be able to generate a build system. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
adamdebreceni commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683418287
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
oh I see, that makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393962#comment-17393962 ] Pierre Villard commented on NIFI-9012: -- Anything that pops up about archiving in the content repository? But it'd make sense if you're close to 50% of used space for the content repo. > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: docker_inspect_excerpt.txt, nifi.properties, > thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
lordgamez commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683353174
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set decompression will only run the
"gunzip" operation on the data and the end content will still be encapsulated
in tar. That is why I followed this same process in the filename changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1156: MINIFICPP-1625 Transfer to failure when Azure blob fails to upload
lordgamez opened a new pull request #1156: URL: https://github.com/apache/nifi-minifi-cpp/pull/1156 `ReadCallback`'s `process` should not fail when the Azure Blob Storage upload fails. The result should be stored and the flowfile should be transferred to the failure relationship. https://issues.apache.org/jira/browse/MINIFICPP-1625 -- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1147: MINIFICPP-1289 add range-v3 and examples
szaszm commented on pull request #1147: URL: https://github.com/apache/nifi-minifi-cpp/pull/1147#issuecomment-893417793 Updated range-v3 from 0.11.0 (latest release) to [c6d7c74](https://github.com/ericniebler/range-v3/commit/c6d7c7480ebdb7f3fcc124c8fdd1b48f7a5adf39), because after rebasing to partial C++20-enabled main, the AppleClang build broke. Related range-v3 issue: https://github.com/ericniebler/range-v3/issues/1604 Please let me know if you still approve the changes, or would rather not depend on a non-release version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9012) InvokeHttps
[
https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393951#comment-17393951
]
Michal Šunka commented on NIFI-9012:
[~pvillard] - Are you interested in anything specific in log files? To be
honest I am not wiling to share those not-anonymized and doing so would take
quite some time. But in the logs for 80 minutes prior first restart and up to
the third restart no error stacktraces are present (aside from those concerning
unavailable site-to-site server during restart).
Here are anonymized [^nifi.properties].
{{Concerning repository:
/opt/nifi/nifi-current$ ls -l | grep repo
drwxr-xr-x. 1026 nifi nifi 20480 Aug 4 12:58 content_repository
drwxr-xr-x. 2 nifi nifi 100 Aug 4 12:58 database_repository
drwxr-xr-x. 4 nifi nifi 52 Aug 5 11:57 flowfile_repository
drwxr-xr-x. 4 nifi nifi 8192 Aug 5 11:55 provenance_repository
}}
But it is from now when all is OK. Or you had something different in mind?
And container as a whole has no disk limit I am aware of. My other instance
(with similar settings, version 1.12.1) had to made 10 GB of logs to stop
working due to the no-space errors. Some container settings:
[^docker_inspect_excerpt.txt]
> InvokeHttps
> ---
>
> Key: NIFI-9012
> URL: https://issues.apache.org/jira/browse/NIFI-9012
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.14.0
> Environment: Deckerized NiFi with container from Docker Hub, no
> custom processors
>Reporter: Michal Šunka
>Priority: Major
> Attachments: docker_inspect_excerpt.txt, nifi.properties,
> thread-dump.txt, thread-dump2.txt
>
>
> I was running a NiFi version 1.12.1 without any problems with this regard.
> Yesterday I updated to version 1.14.0 (really wanted to get fix
> https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse
> state.
> I use InvokeHTTP processor to self-connect to NiFi API to get count of
> flowfiles being processed by given processor group.
> However this InvokeHTTP started getting stuck. Processor would just spin up a
> new thread and this will sit there indefinitely (six hours at-least),
> blocking incoming flowfile. After forceful termination of processor (simple
> stop is not enough) the flowfile would get released and next start will
> consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10
> seconds and idle 1 minute.) And of course log is clear of any errors which
> seems to be connected to this...
> I had noticed similar issue also with other processors, namely MergeContent
> processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes
> (merge bin max age is 5 seconds). After forced termination all starts working
> (for some time - a few dozens of minutes).
> Oh and in all cases the thread hangs in terminating state (red thread icon
> with a digit in parenthesis; the digit sits there all the time and wont
> disappear). Only restart of whole NiFi clears the issue. For some time.
>
> Maybe some disk I/O issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Šunka updated NIFI-9012: --- Attachment: docker_inspect_excerpt.txt > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: docker_inspect_excerpt.txt, nifi.properties, > thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1625) PutAzureBlobStorage processor throws exception when upload fails
Gábor Gyimesi created MINIFICPP-1625: Summary: PutAzureBlobStorage processor throws exception when upload fails Key: MINIFICPP-1625 URL: https://issues.apache.org/jira/browse/MINIFICPP-1625 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Gábor Gyimesi Assignee: Gábor Gyimesi The PutAzureBlobStorage processor's ReadCallback returns and error code when the upload fails which causes a FILE_OPERATION_EXCEPTION to be thrown. This should only happen if we fail to read the flowfile data. In case of upload failure the error result should be stored and the flowfile shall be transferred to failure. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1624) Allow building in docker without up-to-date compilers on the host
Marton Szasz created MINIFICPP-1624: --- Summary: Allow building in docker without up-to-date compilers on the host Key: MINIFICPP-1624 URL: https://issues.apache.org/jira/browse/MINIFICPP-1624 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Marton Szasz Assignee: Marton Szasz After MINIFICPP-1425, there is no reason to require C++20-compatible compilers on hosts that are only used to launch docker containers for building. Please note that cmake still needs _some_ compiler and generator target (e.g. make or ninja) to be installed on the host to be able to generate a build system. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4943: NIFI-5886: Add RPM profile for NiFi Toolkit
asfgit closed pull request #4943: URL: https://github.com/apache/nifi/pull/4943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni edited a comment on pull request #1152: MINIFICPP-1593 ProcFsMonitor to monitor /proc pseduo filesystem
adamdebreceni edited a comment on pull request #1152: URL: https://github.com/apache/nifi-minifi-cpp/pull/1152#issuecomment-893388030 IMO we should wait to introduce new extensions until the dynamic library refactor is merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5190: NIFI-7947: Add directory deletion functionality in DeleteAzureDataLakeStorage
asfgit closed pull request #5190: URL: https://github.com/apache/nifi/pull/5190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4943: NIFI-5886: Add RPM profile for NiFi Toolkit
pvillard31 commented on pull request #4943: URL: https://github.com/apache/nifi/pull/4943#issuecomment-893387737 Merged, thanks @yoshiata @kotarot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #1152: MINIFICPP-1593 ProcFsMonitor to monitor /proc pseduo filesystem
adamdebreceni commented on pull request #1152: URL: https://github.com/apache/nifi-minifi-cpp/pull/1152#issuecomment-893388030 IMO we should wait to introduce new extension until the dynamic library refactor is merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5256: NIFI-8951 - fixed scale vs precision reference in QueryDatabaseTable …
mattyb149 commented on pull request #5256: URL: https://github.com/apache/nifi/pull/5256#issuecomment-892657030 +1 LGTM, thanks for the fix (and Joe for the review)! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5281: NIFI-9008 Added Jetty modules to managed dependencies
asfgit closed pull request #5281: URL: https://github.com/apache/nifi/pull/5281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8128) RedisDistributedMapCacheClientService doesn't work with password protected sentinel
[
https://issues.apache.org/jira/browse/NIFI-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393914#comment-17393914
]
Jens M Kofoed commented on NIFI-8128:
-
Hi [~DamD]
I have a working 3 node Redis Sentinel cluster running with password. I had to
set the same password for both redis and sentinel. But is is working. See
config below:
redis.conf
bind 192.168.1.1 127.0.0.1
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised auto
pidfile "/var/run/redis/redis-server.pid"
loglevel notice
logfile "/var/log/redis/redis-server.log"
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 1
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.rdb"
dir "/var/lib/redis"
masterauth "SOMEPASSWORD"
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
min-replicas-to-write 1
min-replicas-max-lag 10
requirepass "SOMEPASSWORD"
maxmemory-policy allkeys-lru
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
slowlog-log-slower-than 1
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
replicaof 192.168.1.2 6379
sentinel.conf
daemonize yes
pidfile "/var/run/sentinel/redis-sentinel.pid"
logfile "/var/log/redis/redis-sentinel.log"
bind 192.168.1.1
protected-mode yes
port 26379
dir "/var/lib/redis"
sentinel myid 01c53508d72fc00fc8455563f9665c6276ff901c
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 192.168.1.2 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 1
sentinel auth-pass mymaster SOMEPASSWORD
sentinel config-epoch mymaster 44
sentinel leader-epoch mymaster 44
sentinel known-replica mymaster 192.168.1.1 6379
sentinel known-replica mymaster 192.168.1.3 6379
sentinel known-sentinel mymaster 192.168.1.1 26379
5d9e486fcf38a7899a84034b9207c15483bebc8d
sentinel known-sentinel mymaster 192.168.1.2 26379
f8692ee1c25cc04a1acd8771f82fe7cb1b256027
sentinel current-epoch 44
> RedisDistributedMapCacheClientService doesn't work with password protected
> sentinel
> ---
>
> Key: NIFI-8128
> URL: https://issues.apache.org/jira/browse/NIFI-8128
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: DEOM Damien
>Priority: Major
>
>
> RedisDistributedMapCacheClientService doesn't work with password protected
> sentinel .
> Standalone mode with password works fine.
> Removing password in sentinel solves the issue
>
>
> NB: if redis has authentification, Nifi expects the password to be that of
> redis, not that of sentinel
> If redis has pwd but not sentinel, and no pwd is specified in nifi, we get
> this message:
>
> {{org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisException: Could not get a resource from
> the pool}}
>
> If both redis and sentinel have the same pwd, we get this message:
>
> {{ failed to process session due to All sentinels down, cannot determine
> where is mymaster master is running...; Processor Administratively Yielded
> for 1 sec: redis.clients.jedis.exceptions.JedisConnectionException: All
> sentinels down, cannot determine where is mymaster master is running...}}
>
> The documentation should be updated to facilitate the establishment of a
> working, secured cluster.
>
> Original discussion
> [https://stackoverflow.com/questions/65412299/nifi-redis-sentinel-integration]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Šunka updated NIFI-9012: --- Attachment: nifi.properties > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: nifi.properties, thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] adenes commented on pull request #5259: NIFI-8955 Add Max Connection Lifetime property to Hive(_1_1)ConnectionPool CS
adenes commented on pull request #5259: URL: https://github.com/apache/nifi/pull/5259#issuecomment-892950171 > Thanks for making the adjustments and writing up the additional issues @adenes. The improvements here look good, I noted one additional adjustment to reuse the static default value. The change should be made in both files. Good point @exceptionfactory , thanks, I have updated the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5271: NIFI-8957 NiFi Registry - Possibility to set a description when creating a bucket
asfgit closed pull request #5271: URL: https://github.com/apache/nifi/pull/5271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1143: MINIFICPP-1522 Log property name while parsing config in case of property validation failure
fgerlits closed pull request #1143: URL: https://github.com/apache/nifi-minifi-cpp/pull/1143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4773: NIFI-8161 NiFi EL: migration from SimpleDateFormat to DateTimeFormatter
github-actions[bot] commented on pull request #4773: URL: https://github.com/apache/nifi/pull/4773#issuecomment-893060352 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5276: NIFI-8990 Upgraded Groovy to 2.5.14
mattyb149 commented on pull request #5276: URL: https://github.com/apache/nifi/pull/5276#issuecomment-892652267 +1 LGTM, thanks for the upgrade (the Tinkerpop stuff was sorely needed!) Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #5271: NIFI-8957 NiFi Registry - Possibility to set a description when creating a bucket
pvillard31 commented on pull request #5271: URL: https://github.com/apache/nifi/pull/5271#issuecomment-893326770 Merged, thanks @adenes @markobean @mtien-apache -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #5281: NIFI-9008 Added Jetty modules to managed dependencies
pvillard31 commented on pull request #5281: URL: https://github.com/apache/nifi/pull/5281#issuecomment-893329612 Merged, thanks @exceptionfactory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #5276: NIFI-8990 Upgraded Groovy to 2.5.14
mattyb149 closed pull request #5276: URL: https://github.com/apache/nifi/pull/5276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393877#comment-17393877 ] Michal Šunka commented on NIFI-9012: I have found two such dumps from my own debugging attempts, but I cannot give any reasonable info about when they were taken (aside that after I noticed some blocked threads). Sorry. If it happens again, I will do the dumps again. And with better info hopefully :) > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393880#comment-17393880 ] Pierre Villard commented on NIFI-9012: -- Can you share/look at nifi-app.logs? Also can you share nifi.properties? and the amount of used/available space for your content repo? > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
adamdebreceni commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683343728
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
using the `encapsulateInTar_` during decompression seems strange,
shouldn't we infer the need for "de-tar-ing" only from the extension?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-5886) Add RPM profile for NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-5886: - Fix Version/s: 1.15.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add RPM profile for NiFi Toolkit > > > Key: NIFI-5886 > URL: https://issues.apache.org/jira/browse/NIFI-5886 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Yoshiaki Takahashi >Assignee: Yoshiaki Takahashi >Priority: Major > Fix For: 1.15.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > RPM for NiFi Toolkit is useful for easy installation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Šunka updated NIFI-9012: --- Attachment: thread-dump2.txt thread-dump.txt > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > Attachments: thread-dump.txt, thread-dump2.txt > > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #5278: NIFI-8991: reduce Node requirement across modules to a single version
asfgit closed pull request #5278: URL: https://github.com/apache/nifi/pull/5278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-5886) Add RPM profile for NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393873#comment-17393873 ] ASF subversion and git services commented on NIFI-5886: --- Commit 4fa357ed9e9f0d1f61a98910eec2e9e193aebc87 in nifi's branch refs/heads/main from Yoshiaki Takahashi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4fa357e ] NIFI-5886: Add RPM profile for NiFi Toolkit Co-authored-by: Kotaro Terada Signed-off-by: Pierre Villard This closes #4943. > Add RPM profile for NiFi Toolkit > > > Key: NIFI-5886 > URL: https://issues.apache.org/jira/browse/NIFI-5886 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Yoshiaki Takahashi >Assignee: Yoshiaki Takahashi >Priority: Major > Time Spent: 2h 50m > Remaining Estimate: 0h > > RPM for NiFi Toolkit is useful for easy installation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] yoshiata commented on pull request #4943: NIFI-5886: Add RPM profile for NiFi Toolkit
yoshiata commented on pull request #4943: URL: https://github.com/apache/nifi/pull/4943#issuecomment-893196938 @kotarot @pvillard31 Thank you for reviewing the code and taking care of the label! Fixed the code. Please review again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1146: MINIFICPP-1617 Update CONTRIB.md with current practice
fgerlits closed pull request #1146: URL: https://github.com/apache/nifi-minifi-cpp/pull/1146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1138: MINIFICPP-1471 - Build extensions as dynamic libraries and dynamically load them
adamdebreceni commented on a change in pull request #1138:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1138#discussion_r683288063
##
File path: libminifi/include/core/state/nodes/DeviceInformation.h
##
@@ -39,6 +37,10 @@
#else
#pragma comment(lib, "iphlpapi.lib")
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN 1
+#endif
Review comment:
it seems you yourself have already added it in #967, but did not remove
them from source/header files, we should remove all of them in a separate PR
##
File path: libminifi/include/core/state/nodes/DeviceInformation.h
##
@@ -39,6 +37,10 @@
#else
#pragma comment(lib, "iphlpapi.lib")
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN 1
+#endif
Review comment:
~it seems you yourself have already added it in #967~ somebody have
already added it, but did not remove them from source/header files, we should
remove all of them in a separate PR
edit: you just modified the line
##
File path: libminifi/include/core/state/nodes/DeviceInformation.h
##
@@ -39,6 +37,10 @@
#else
#pragma comment(lib, "iphlpapi.lib")
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN 1
+#endif
Review comment:
removed this one
##
File path: libminifi/test/unit/MemoryUsageTest.cpp
##
@@ -16,6 +16,9 @@
* limitations under the License.
*/
+// including the extensions messes up the memory usage
+#define EXTENSION_LIST ""
Review comment:
updated comment
##
File path: libminifi/src/utils/file/FileMatcher.cpp
##
@@ -0,0 +1,294 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "utils/file/FileMatcher.h"
+#include "utils/file/FileUtils.h"
+#include "utils/StringUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+namespace file {
+
+std::shared_ptr FileMatcher::FilePattern::logger_ =
logging::LoggerFactory::getLogger();
+std::shared_ptr FileMatcher::logger_ =
logging::LoggerFactory::getLogger();
+
+static bool isGlobPattern(const std::string& pattern) {
+ return pattern.find_first_of("?*") != std::string::npos;
+}
+
+static std::vector split(const std::string& str, const
std::vector& delimiters) {
+ std::vector result;
+
+ size_t prev_delim_end = 0;
+ size_t next_delim_begin = std::string::npos;
+ do {
+for (const auto& delim : delimiters) {
+ next_delim_begin = str.find(delim, prev_delim_end);
+ if (next_delim_begin != std::string::npos) {
+result.push_back(str.substr(prev_delim_end, next_delim_begin -
prev_delim_end));
+prev_delim_end = next_delim_begin + delim.length();
+break;
+ }
+}
+ } while (next_delim_begin != std::string::npos);
+ result.push_back(str.substr(prev_delim_end));
+ return result;
+}
+
+#ifdef WIN32
+static const std::vector path_separators{"/", "\\"};
+#else
+static const std::vector path_separators{"/"};
+#endif
+
+optional
FileMatcher::FilePattern::fromPattern(std::string pattern) {
+ pattern = utils::StringUtils::trim(pattern);
+ bool excluding = false;
+ if (!pattern.empty() && pattern[0] == '!') {
+excluding = true;
+pattern = utils::StringUtils::trim(pattern.substr(1));
+ }
+ if (pattern.empty()) {
+logger_->log_error("Empty pattern");
+return nullopt;
+ }
+ std::string exe_dir = get_executable_dir();
+ if (exe_dir.empty() && !isAbsolutePath(pattern.c_str())) {
+logger_->log_error("Couldn't determine executable dir, relative pattern
'%s' not supported", pattern);
+return nullopt;
+ }
+ pattern = resolve(exe_dir, pattern);
+ auto segments = split(pattern, path_separators);
+ gsl_Expects(!segments.empty());
+ auto file_pattern = segments.back();
+ if (file_pattern == "**") {
+file_pattern = "*";
+ } else {
+segments.pop_back();
+ }
+ if (file_pattern == "." || file_pattern == "..") {
+logger_->log_error("Invalid file pattern '%s'", file_pattern);
+return nullopt;
+ }
+ bool after_wildcard = false;
+ for (const auto& segment : segments) {
+if (after_wildcard && segment == "..") {
+ logger_->log_error("Parent accessor is not supported after wildcards");
+ return
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5195: NIFI-8752: Automatic diagnostic at NiFi restart/stop
exceptionfactory commented on a change in pull request #5195:
URL: https://github.com/apache/nifi/pull/5195#discussion_r682574278
##
File path: nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
##
@@ -281,6 +285,23 @@ public static void main(String[] args) throws IOException,
InterruptedException
}
}
+private void runDiagnostics() throws IOException {
+final DiagnosticContext diagnosticContext = new DiagnosticContext();
Review comment:
The `RunNiFi` class has a `bootstrapConfigFile` property referencing
`bootstrap.conf`. The `DiagnosticContext` should be refactored to use the
existing file reference and similar approach to loading properties as other
methods in this class. Refactoring also provides an opportunity to separate
configuration properties from evaluation methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni closed pull request #1150: MINIFICPP-1611 Add #include in the patch
adamdebreceni closed pull request #1150: URL: https://github.com/apache/nifi-minifi-cpp/pull/1150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #5280: NIFI-8996: Close JDBC statements in PutHive*QL processors.
mattyb149 closed pull request #5280: URL: https://github.com/apache/nifi/pull/5280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
lordgamez commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683353174
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression only the "gunzip"
operation will be run on the data and the end content will still be
encapsulated in tar. That is why I followed this same process in the filename
changes.
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression will only run the
"gunzip" operation will be run on the data and the end content will still be
encapsulated in tar. That is why I followed this same process in the filename
changes.
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression will only run the
"gunzip" operation on the data and the end content will still be encapsulated
in tar. That is why I followed this same process in the filename changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7947) DeleteAzureDataLakeStorage Processor cannot delete non empty folder
[
https://issues.apache.org/jira/browse/NIFI-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-7947:
--
Fix Version/s: 1.15.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> DeleteAzureDataLakeStorage Processor cannot delete non empty folder
> ---
>
> Key: NIFI-7947
> URL: https://issues.apache.org/jira/browse/NIFI-7947
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Stan Antyufeev
>Assignee: Lehel Boér
>Priority: Major
> Fix For: 1.15.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The current implementation of DeleteAzureDataLakeStorage version
> 1.11.4.3.5.1.19-1 cannot delete non empty folder if you specify a folder name
> in "File name" field and leave empty "Directory name" field.
> The error message is
> DeleteAzureDataLakeStorage[id=50e969ee-0175-1000--31e844c3] Failed to
> delete the specified file from Azure Data Lake Storage:
> com.azure.storage.file.datalake.models.DataLakeStorageException: Status code
> 409, "\{"error":{"code":"DirectoryNotEmpty","message":"The recursive query
> parameter value must be true to delete a non-empty
> directory.\nRequestId:525ebddf-c01f-00cf-19a8-abcb9400\nTime:2020-10-26T15:01:35.0517746Z"}}
>
> The azcopy command has extra flag "-recursive" to delete non-empty folders:
> [https://docs.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-remove]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7947) DeleteAzureDataLakeStorage Processor cannot delete non empty folder
[
https://issues.apache.org/jira/browse/NIFI-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393847#comment-17393847
]
ASF subversion and git services commented on NIFI-7947:
---
Commit 9bf3b5503434e8a1dad5453b54a38dfaf079467c in nifi's branch
refs/heads/main from Lehel Boér
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9bf3b55 ]
NIFI-7947: Add directory deletion functionality in DeleteAzureDataLakeStorage
This closes #5190.
Signed-off-by: Peter Turcsanyi
> DeleteAzureDataLakeStorage Processor cannot delete non empty folder
> ---
>
> Key: NIFI-7947
> URL: https://issues.apache.org/jira/browse/NIFI-7947
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Stan Antyufeev
>Assignee: Lehel Boér
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The current implementation of DeleteAzureDataLakeStorage version
> 1.11.4.3.5.1.19-1 cannot delete non empty folder if you specify a folder name
> in "File name" field and leave empty "Directory name" field.
> The error message is
> DeleteAzureDataLakeStorage[id=50e969ee-0175-1000--31e844c3] Failed to
> delete the specified file from Azure Data Lake Storage:
> com.azure.storage.file.datalake.models.DataLakeStorageException: Status code
> 409, "\{"error":{"code":"DirectoryNotEmpty","message":"The recursive query
> parameter value must be true to delete a non-empty
> directory.\nRequestId:525ebddf-c01f-00cf-19a8-abcb9400\nTime:2020-10-26T15:01:35.0517746Z"}}
>
> The azcopy command has extra flag "-recursive" to delete non-empty folders:
> [https://docs.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-remove]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markobean commented on pull request #5247: NIFI-8927 - Add option to start/stop all controllers
markobean commented on pull request #5247: URL: https://github.com/apache/nifi/pull/5247#issuecomment-892718873 I suggest you update references to the phrasing "controllers" to "controller services". This is most important in the user-facing UI, but also will add clarity in the code as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mtien-apache commented on a change in pull request #5271: NIFI-8957 NiFi Registry - Possibility to set a description when creating a bucket
mtien-apache commented on a change in pull request #5271:
URL: https://github.com/apache/nifi/pull/5271#discussion_r683072068
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main/webapp/components/administration/workflow/sidenav/manage-bucket/nf-registry-manage-bucket.js
##
@@ -407,11 +411,12 @@ NfRegistryManageBucket.prototype = {
*
* @param username
*/
Review comment:
I know this was already here, but would you update the documentation
`@param` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393832#comment-17393832 ] Peter Turcsanyi commented on NIFI-9012: --- [~RaoCZ] When the issue occurs again, could you please create thread dumps (before restarting NiFi) and attach them to the jira? 3 dumps with 3 minute intervals would be fine. You can use "nifi.sh dump [dump1.out]" command. > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 closed pull request #5256: NIFI-8951 - fixed scale vs precision reference in QueryDatabaseTable …
mattyb149 closed pull request #5256: URL: https://github.com/apache/nifi/pull/5256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1151: MINIFICPP-1621 fix docker focal build
lordgamez commented on a change in pull request #1151:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1151#discussion_r682577916
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
COPY . ${MINIFI_BASE_DIR}
FROM build_deps AS release
ARG ENABLE_JNI
-# Perform the build
-RUN cd $MINIFI_BASE_DIR \
- && ./bootstrap.sh -e -t \
- && rm -rf build \
- && mkdir build \
- && cd build \
- && cmake -DUSE_SHARED_LIBS= -DENABLE_MQTT=ON -DENABLE_LIBRDKAFKA=ON
-DPORTABLE=ON -DENABLE_COAP=ON -DCMAKE_BUILD_TYPE=Release -DSKIP_TESTS=true
-DENABLE_JNI=$ENABLE_JNI .. \
- && make -j$(nproc) package
+
+# Run bootstrap
+RUN cd $MINIFI_BASE_DIR && ./bootstrap.sh -t
+
+ENV CC gcc-11
+ENV CXX g++-11
+
+# Build
+RUN cd $MINIFI_BASE_DIR/build \
Review comment:
ENV variables could be moved up and the two RUN commands merged into a
single one
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
Review comment:
These commands could be merged into a single RUN command
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
Review comment:
Usually it's recommended to minimize the layer count and as every
command in a Dockerfile creates a new layer it is good practice to merge
multiple RUN commands to have less layers.
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
Review comment:
Only RUN commands can be chained in a bash like manner with `&&` ENV and
COPY don't really have a similar option
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
Review comment:
Only RUN commands can be chained in a bash like manner with `&&` (as it
is just a command run inside the container) ENV and COPY don't really have a
similar option
##
File path: docker/focal/Dockerfile
##
@@ -29,19 +29,22 @@ ENV MINIFI_HOME
$MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN apt update
-RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
+RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk
openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake
software-properties-common
RUN mkdir -p $MINIFI_BASE_DIR
Review comment:
But if the question was regarding the creation of the layers, yes all of
those create new layers.
##
File path:
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1144: MINIFICPP-1429 clean up pre-c++17 workarounds
fgerlits closed pull request #1144: URL: https://github.com/apache/nifi-minifi-cpp/pull/1144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] eduardofontes commented on pull request #5247: NIFI-8927 - Add option to start/stop all controllers
eduardofontes commented on pull request #5247: URL: https://github.com/apache/nifi/pull/5247#issuecomment-892931518 @markobean Done! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5280: NIFI-8996: Close JDBC statements in PutHive*QL processors.
mattyb149 commented on pull request #5280: URL: https://github.com/apache/nifi/pull/5280#issuecomment-892647475 +1 LGTM, thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1151: MINIFICPP-1621 fix docker focal build
szaszm commented on a change in pull request #1151: URL: https://github.com/apache/nifi-minifi-cpp/pull/1151#discussion_r682584864 ## File path: docker/focal/Dockerfile ## @@ -29,19 +29,22 @@ ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN apt update -RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake +RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake software-properties-common RUN mkdir -p $MINIFI_BASE_DIR Review comment: Would it be better that way? ## File path: docker/focal/Dockerfile ## @@ -29,19 +29,22 @@ ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN apt update -RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake +RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake software-properties-common RUN mkdir -p $MINIFI_BASE_DIR Review comment: Would it be better that way? If so, why? I'm not a docker magician. :) ## File path: docker/focal/Dockerfile ## @@ -29,19 +29,22 @@ ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN apt update -RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake +RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake software-properties-common RUN mkdir -p $MINIFI_BASE_DIR Review comment: Does this include ENV and COPY, or only RUN? ## File path: docker/focal/Dockerfile ## @@ -29,19 +29,22 @@ ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN apt update -RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake +RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake software-properties-common RUN mkdir -p $MINIFI_BASE_DIR Review comment: I mean do they also introduce separate layers? Should I keep DEBIAN_FRONTEND as a prefix to the command or separate it to an ENV command? ## File path: docker/focal/Dockerfile ## @@ -29,19 +29,22 @@ ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN apt update -RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake +RUN DEBIAN_FRONTEND="noninteractive" apt install -y openjdk-8-jdk openjdk-8-source python3.9-dev sudo git maven autogen autoconf automake cmake software-properties-common RUN mkdir -p $MINIFI_BASE_DIR Review comment: I mean do they also introduce separate layers? Should I keep DEBIAN_FRONTEND as a prefix to the RUN command or separate it to an ENV command? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
lordgamez commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683353174
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression will only run the
"gunzip" operation will be run on the data and the end content will still be
encapsulated in tar. That is why I followed this same process in the filename
changes.
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression will only run the
"gunzip" operation on the data and the end content will still be encapsulated
in tar. That is why I followed this same process in the filename changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
lordgamez commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683353174
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
Currently the "de-tar-ing" depends on the `Encapsulate in TAR` property
in the processor, so if that is not set in decompression only the "gunzip"
operation will be run on the data and the end content will still be
encapsulated in tar. That is why I followed this same process in the filename
changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1153: MINIFICPP-1622 Fix filename update of CompressContent in case of tar.gz
adamdebreceni commented on a change in pull request #1153:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1153#discussion_r683343728
##
File path: extensions/libarchive/CompressContent.cpp
##
@@ -191,14 +194,20 @@ void CompressContent::processFlowFile(const
std::shared_ptr& flo
if (compressMode_ == CompressionMode::Compress) {
session->putAttribute(result, core::SpecialFlowAttribute::MIME_TYPE,
mimeType);
if (updateFileName_) {
+if (encapsulateInTar_) {
+ fileName = fileName + TAR_EXT;
+}
fileName = fileName + fileExtension;
session->putAttribute(result, core::SpecialFlowAttribute::FILENAME,
fileName);
}
} else {
session->removeAttribute(result, core::SpecialFlowAttribute::MIME_TYPE);
if (updateFileName_) {
-if (fileName.size() >= fileExtension.size() &&
fileName.compare(fileName.size() - fileExtension.size(), fileExtension.size(),
fileExtension) == 0) {
+if (utils::StringUtils::endsWith(fileName, fileExtension)) {
fileName = fileName.substr(0, fileName.size() -
fileExtension.size());
+ if (encapsulateInTar_ && utils::StringUtils::endsWith(fileName,
TAR_EXT)) {
Review comment:
using the `encapsulateInTar_` during decompression seems strange,
shouldn't we infer the need for "de-tar-ing" only from the extension?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1155: MINIFICPP-1619 Make sure serialized records expire in DBProvenanceRepositoryTests
lordgamez opened a new pull request #1155: URL: https://github.com/apache/nifi-minifi-cpp/pull/1155 DBProvenanceRepositoryTests transiently failed due to not waiting enough time to have the serialized records to expire in the test. The failure does not seem to be connected to a specific change as it seems to transiently fail with commits from a few months ago, but the failures seem to increase in the past week. Increasing the wait time from 1,5 to 2 seconds solves the issue. https://issues.apache.org/jira/browse/MINIFICPP-1619 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9012) InvokeHttps
[ https://issues.apache.org/jira/browse/NIFI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393779#comment-17393779 ] Michal Šunka commented on NIFI-9012: Werid, it seems that some third restart of the container did the job and nothing seems to get stuck anymore... Werid. However I let this issue to stay open to let you know this had happened... > InvokeHttps > --- > > Key: NIFI-9012 > URL: https://issues.apache.org/jira/browse/NIFI-9012 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0 > Environment: Deckerized NiFi with container from Docker Hub, no > custom processors >Reporter: Michal Šunka >Priority: Major > > I was running a NiFi version 1.12.1 without any problems with this regard. > Yesterday I updated to version 1.14.0 (really wanted to get fix > https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse > state. > I use InvokeHTTP processor to self-connect to NiFi API to get count of > flowfiles being processed by given processor group. > However this InvokeHTTP started getting stuck. Processor would just spin up a > new thread and this will sit there indefinitely (six hours at-least), > blocking incoming flowfile. After forceful termination of processor (simple > stop is not enough) the flowfile would get released and next start will > consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 > seconds and idle 1 minute.) And of course log is clear of any errors which > seems to be connected to this... > I had noticed similar issue also with other processors, namely MergeContent > processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes > (merge bin max age is 5 seconds). After forced termination all starts working > (for some time - a few dozens of minutes). > Oh and in all cases the thread hangs in terminating state (red thread icon > with a digit in parenthesis; the digit sits there all the time and wont > disappear). Only restart of whole NiFi clears the issue. For some time. > > Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-9012) InvokeHttps
Michal Šunka created NIFI-9012: -- Summary: InvokeHttps Key: NIFI-9012 URL: https://issues.apache.org/jira/browse/NIFI-9012 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.14.0 Environment: Deckerized NiFi with container from Docker Hub, no custom processors Reporter: Michal Šunka I was running a NiFi version 1.12.1 without any problems with this regard. Yesterday I updated to version 1.14.0 (really wanted to get fix https://issues.apache.org/jira/browse/NIFI-7849). But now I am in a way worse state. I use InvokeHTTP processor to self-connect to NiFi API to get count of flowfiles being processed by given processor group. However this InvokeHTTP started getting stuck. Processor would just spin up a new thread and this will sit there indefinitely (six hours at-least), blocking incoming flowfile. After forceful termination of processor (simple stop is not enough) the flowfile would get released and next start will consume the flowfile just fine. (Timeouts are: connection 5 seconds, read 10 seconds and idle 1 minute.) And of course log is clear of any errors which seems to be connected to this... I had noticed similar issue also with other processors, namely MergeContent processor or ExecuteSQLProcessor - e.g. flowfile gets stuck for 15 minutes (merge bin max age is 5 seconds). After forced termination all starts working (for some time - a few dozens of minutes). Oh and in all cases the thread hangs in terminating state (red thread icon with a digit in parenthesis; the digit sits there all the time and wont disappear). Only restart of whole NiFi clears the issue. For some time. Maybe some disk I/O issue? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1138: MINIFICPP-1471 - Build extensions as dynamic libraries and dynamically load them
adamdebreceni commented on a change in pull request #1138:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1138#discussion_r683314053
##
File path: libminifi/src/utils/file/FileMatcher.cpp
##
@@ -0,0 +1,294 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "utils/file/FileMatcher.h"
+#include "utils/file/FileUtils.h"
+#include "utils/StringUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+namespace file {
+
+std::shared_ptr FileMatcher::FilePattern::logger_ =
logging::LoggerFactory::getLogger();
+std::shared_ptr FileMatcher::logger_ =
logging::LoggerFactory::getLogger();
+
+static bool isGlobPattern(const std::string& pattern) {
+ return pattern.find_first_of("?*") != std::string::npos;
+}
+
+static std::vector split(const std::string& str, const
std::vector& delimiters) {
+ std::vector result;
+
+ size_t prev_delim_end = 0;
+ size_t next_delim_begin = std::string::npos;
+ do {
+for (const auto& delim : delimiters) {
+ next_delim_begin = str.find(delim, prev_delim_end);
+ if (next_delim_begin != std::string::npos) {
+result.push_back(str.substr(prev_delim_end, next_delim_begin -
prev_delim_end));
+prev_delim_end = next_delim_begin + delim.length();
+break;
+ }
+}
+ } while (next_delim_begin != std::string::npos);
+ result.push_back(str.substr(prev_delim_end));
+ return result;
+}
+
+#ifdef WIN32
+static const std::vector path_separators{"/", "\\"};
+#else
+static const std::vector path_separators{"/"};
+#endif
+
+optional
FileMatcher::FilePattern::fromPattern(std::string pattern) {
+ pattern = utils::StringUtils::trim(pattern);
+ bool excluding = false;
+ if (!pattern.empty() && pattern[0] == '!') {
+excluding = true;
+pattern = utils::StringUtils::trim(pattern.substr(1));
+ }
+ if (pattern.empty()) {
+logger_->log_error("Empty pattern");
+return nullopt;
+ }
+ std::string exe_dir = get_executable_dir();
+ if (exe_dir.empty() && !isAbsolutePath(pattern.c_str())) {
+logger_->log_error("Couldn't determine executable dir, relative pattern
'%s' not supported", pattern);
+return nullopt;
+ }
+ pattern = resolve(exe_dir, pattern);
+ auto segments = split(pattern, path_separators);
+ gsl_Expects(!segments.empty());
+ auto file_pattern = segments.back();
+ if (file_pattern == "**") {
+file_pattern = "*";
+ } else {
+segments.pop_back();
+ }
+ if (file_pattern == "." || file_pattern == "..") {
+logger_->log_error("Invalid file pattern '%s'", file_pattern);
+return nullopt;
+ }
+ bool after_wildcard = false;
+ for (const auto& segment : segments) {
+if (after_wildcard && segment == "..") {
+ logger_->log_error("Parent accessor is not supported after wildcards");
+ return nullopt;
+}
+if (isGlobPattern(segment)) {
+ after_wildcard = true;
+}
+ }
+ return FilePattern(segments, file_pattern, excluding);
+}
+
+std::string FileMatcher::FilePattern::getBaseDirectory() const {
+ std::string base_dir;
+ for (const auto& segment : directory_segments_) {
+// ignore segments at or after wildcards
+if (isGlobPattern(segment)) {
+ break;
+}
+base_dir += segment + get_separator();
+ }
+ return base_dir;
+}
+
+FileMatcher::FileMatcher(const std::string ) {
+ for (auto&& pattern : split(patterns, {","})) {
+if (auto&& p = FilePattern::fromPattern(pattern)) {
+ patterns_.push_back(std::move(p.value()));
+}
+ }
+}
+
+template
+static bool advance_if_not_equal(It& it, const It& end) {
+ if (it == end) {
+return false;
+ }
+ ++it;
+ return true;
+}
+
+static bool is_this_dir(const std::string& dir) {
+ return dir.empty() || dir == ".";
+}
+
+template
+static void skip_if(It& it, const It& end, const Fn& fn) {
+ while (it != end && fn(*it)) {
+++it;
+ }
+}
+
+static bool matchGlob(std::string::const_iterator pattern_it,
std::string::const_iterator pattern_end, std::string::const_iterator value_it,
std::string::const_iterator value_end) {
Review comment:
it seems that on my machine `Apple clang version 11.0.3` does not
support that constructor even with the `-std=c++2a`
[jira] [Resolved] (NIFI-9008) Add Jetty modules to Managed Dependencies
[
https://issues.apache.org/jira/browse/NIFI-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-9008.
--
Resolution: Fixed
> Add Jetty modules to Managed Dependencies
> -
>
> Key: NIFI-9008
> URL: https://issues.apache.org/jira/browse/NIFI-9008
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, NiFi Registry
>Affects Versions: 1.15.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.15.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The NiFi main Maven configuration includes managed dependencies for a number
> of Jetty modules such as {{jetty-server}} and {{jetty-annotations}}. This
> approach ensures that all child modules depend on the version specified in
> the main configuration.
> Some components, such as Hive 3 and Solr, depend on additional Jetty modules,
> resulting a mixture of versions in bundled NAR files. The main Maven
> configuration should be updated with additional Jetty modules to ensure
> consistent versioning in dependent bundles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-9008) Add Jetty modules to Managed Dependencies
[
https://issues.apache.org/jira/browse/NIFI-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-9008:
-
Fix Version/s: 1.15.0
> Add Jetty modules to Managed Dependencies
> -
>
> Key: NIFI-9008
> URL: https://issues.apache.org/jira/browse/NIFI-9008
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, NiFi Registry
>Affects Versions: 1.15.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.15.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The NiFi main Maven configuration includes managed dependencies for a number
> of Jetty modules such as {{jetty-server}} and {{jetty-annotations}}. This
> approach ensures that all child modules depend on the version specified in
> the main configuration.
> Some components, such as Hive 3 and Solr, depend on additional Jetty modules,
> resulting a mixture of versions in bundled NAR files. The main Maven
> configuration should be updated with additional Jetty modules to ensure
> consistent versioning in dependent bundles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-9008) Add Jetty modules to Managed Dependencies
[
https://issues.apache.org/jira/browse/NIFI-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393769#comment-17393769
]
ASF subversion and git services commented on NIFI-9008:
---
Commit c273b02ebee7994d76771745e686032f3da39449 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c273b02 ]
NIFI-9008 Added Jetty modules to managed dependencies
- Updated OWASP dependency check suppressions with jetty-test-helper
Signed-off-by: Pierre Villard
This closes #5281.
> Add Jetty modules to Managed Dependencies
> -
>
> Key: NIFI-9008
> URL: https://issues.apache.org/jira/browse/NIFI-9008
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, NiFi Registry
>Affects Versions: 1.15.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The NiFi main Maven configuration includes managed dependencies for a number
> of Jetty modules such as {{jetty-server}} and {{jetty-annotations}}. This
> approach ensures that all child modules depend on the version specified in
> the main configuration.
> Some components, such as Hive 3 and Solr, depend on additional Jetty modules,
> resulting a mixture of versions in bundled NAR files. The main Maven
> configuration should be updated with additional Jetty modules to ensure
> consistent versioning in dependent bundles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
