[jira] [Updated] (NIFI-8656) Support expression language for SAS Token in the ADLS Gen2 processors
[ https://issues.apache.org/jira/browse/NIFI-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Frazee updated NIFI-8656: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Support expression language for SAS Token in the ADLS Gen2 processors > - > > Key: NIFI-8656 > URL: https://issues.apache.org/jira/browse/NIFI-8656 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > In use cases where the SAS Token acquired at runtime (e.g. via InvokeHttp), > it would be useful to support EL for SAS Token property to pass the token to > the processor dynamically. > The Blob counterpart already have this feature and the common description of > the property already mentions the security considerations of this scenario. > The same can be used for the ADLS processors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8656) Support expression language for SAS Token in the ADLS Gen2 processors
[ https://issues.apache.org/jira/browse/NIFI-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362072#comment-17362072 ] Joey Frazee commented on NIFI-8656: --- [~joewitt] Not sure where you are now on building the release so I've deliberately left the Fix Version unset. LMK which way it goes and I'll set it. > Support expression language for SAS Token in the ADLS Gen2 processors > - > > Key: NIFI-8656 > URL: https://issues.apache.org/jira/browse/NIFI-8656 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > In use cases where the SAS Token acquired at runtime (e.g. via InvokeHttp), > it would be useful to support EL for SAS Token property to pass the token to > the processor dynamically. > The Blob counterpart already have this feature and the common description of > the property already mentions the security considerations of this scenario. > The same can be used for the ADLS processors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8656) Support expression language for SAS Token in the ADLS Gen2 processors
[ https://issues.apache.org/jira/browse/NIFI-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Frazee updated NIFI-8656: -- Component/s: Extensions > Support expression language for SAS Token in the ADLS Gen2 processors > - > > Key: NIFI-8656 > URL: https://issues.apache.org/jira/browse/NIFI-8656 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > In use cases where the SAS Token acquired at runtime (e.g. via InvokeHttp), > it would be useful to support EL for SAS Token property to pass the token to > the processor dynamically. > The Blob counterpart already have this feature and the common description of > the property already mentions the security considerations of this scenario. > The same can be used for the ADLS processors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8656) Support expression language for SAS Token in the ADLS Gen2 processors
[ https://issues.apache.org/jira/browse/NIFI-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Frazee updated NIFI-8656: -- Affects Version/s: 1.13.2 > Support expression language for SAS Token in the ADLS Gen2 processors > - > > Key: NIFI-8656 > URL: https://issues.apache.org/jira/browse/NIFI-8656 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.13.2 >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > In use cases where the SAS Token acquired at runtime (e.g. via InvokeHttp), > it would be useful to support EL for SAS Token property to pass the token to > the processor dynamically. > The Blob counterpart already have this feature and the common description of > the property already mentions the security considerations of this scenario. > The same can be used for the ADLS processors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8656) Support expression language for SAS Token in the ADLS Gen2 processors
[ https://issues.apache.org/jira/browse/NIFI-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362066#comment-17362066 ] ASF subversion and git services commented on NIFI-8656: --- Commit 1a515ee74a18d1ec881cc028b6d53bddbfbf2f8b in nifi's branch refs/heads/main from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1a515ee ] NIFI-8656 Support EL for SAS Token in the ADLS Gen2 processors This closes #5119 Signed-off-by: Joey Frazee > Support expression language for SAS Token in the ADLS Gen2 processors > - > > Key: NIFI-8656 > URL: https://issues.apache.org/jira/browse/NIFI-8656 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > In use cases where the SAS Token acquired at runtime (e.g. via InvokeHttp), > it would be useful to support EL for SAS Token property to pass the token to > the processor dynamically. > The Blob counterpart already have this feature and the common description of > the property already mentions the security considerations of this scenario. > The same can be used for the ADLS processors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8643: - Fix Version/s: (was: 1.12.1) 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.14.0 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362044#comment-17362044 ] ASF subversion and git services commented on NIFI-8643: --- Commit 6f04b457735bb6d511332cdcc3b0dabf302bf775 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6f04b45 ] NIFI-8643: Correcting nifi.cluster.load.balance.address in properties (#5146) NIFI-8643: Correcting nifi.cluster.load.balance.address in properties > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362043#comment-17362043 ] ASF subversion and git services commented on NIFI-8643: --- Commit 6f04b457735bb6d511332cdcc3b0dabf302bf775 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6f04b45 ] NIFI-8643: Correcting nifi.cluster.load.balance.address in properties (#5146) NIFI-8643: Correcting nifi.cluster.load.balance.address in properties > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8647) Update VersionedPropertyDescriptor to include Resource References
[ https://issues.apache.org/jira/browse/NIFI-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-8647: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Update VersionedPropertyDescriptor to include Resource References > - > > Key: NIFI-8647 > URL: https://issues.apache.org/jira/browse/NIFI-8647 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In NIFI-8206, we added the ability to have Property Descriptors indicate that > they reference some other source of resource, such as a file, URL, etc. We > should update the VersionedPropertyDescriptor to contain this information as > well. This can be useful, as it allows anyone who parses a flow to determine > that there may be some external resource that needs to be made available. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8647) Update VersionedPropertyDescriptor to include Resource References
[ https://issues.apache.org/jira/browse/NIFI-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362041#comment-17362041 ] ASF subversion and git services commented on NIFI-8647: --- Commit 38cf25c7cfc9fc6d80d5f5a77a28b29da53a8fd2 in nifi's branch refs/heads/main from markap14 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=38cf25c ] NIFI-8647: Updated VersionedPropertyDescriptor to include information about which properties are referencing resources, how many resource, and the types of resources; updated NiFiRegistryFlowMapper to perform the mapping. (#5114) > Update VersionedPropertyDescriptor to include Resource References > - > > Key: NIFI-8647 > URL: https://issues.apache.org/jira/browse/NIFI-8647 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In NIFI-8206, we added the ability to have Property Descriptors indicate that > they reference some other source of resource, such as a file, URL, etc. We > should update the VersionedPropertyDescriptor to contain this information as > well. This can be useful, as it allows anyone who parses a flow to determine > that there may be some external resource that needs to be made available. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-8651) Refactor SensitivePropertiesProviderFactory
[ https://issues.apache.org/jira/browse/NIFI-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-8651. Fix Version/s: 1.14.0 Resolution: Fixed > Refactor SensitivePropertiesProviderFactory > --- > > Key: NIFI-8651 > URL: https://issues.apache.org/jira/browse/NIFI-8651 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joseph Gresock >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.14.0 > > Time Spent: 3h > Remaining Estimate: 0h > > In order to facilitate extensibility with SensitivePropertiesProviders (as in > NIFI-5481 and NIFI-8447), we should refactor the > SensitivePropertiesProviderFactory to return the appropriate SPP based on a > "protection scheme". This scheme will correlate to the values of the > `.protected` properties in nifi.properties. > For example, a protectionScheme of `aes/gcm/256` would map to the > AESSensitivePropertiesProvider. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8651) Refactor SensitivePropertiesProviderFactory
[ https://issues.apache.org/jira/browse/NIFI-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362040#comment-17362040 ] ASF subversion and git services commented on NIFI-8651: --- Commit 1ccc4fbb0ff6a4041ddff180ab07a1675308e4a3 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1ccc4fb ] NIFI-8651: Refactor Sensitive Properties Providers for extension This closes #5131 Signed-off-by: David Handermann > Refactor SensitivePropertiesProviderFactory > --- > > Key: NIFI-8651 > URL: https://issues.apache.org/jira/browse/NIFI-8651 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joseph Gresock >Assignee: Joseph Gresock >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > In order to facilitate extensibility with SensitivePropertiesProviders (as in > NIFI-5481 and NIFI-8447), we should refactor the > SensitivePropertiesProviderFactory to return the appropriate SPP based on a > "protection scheme". This scheme will correlate to the values of the > `.protected` properties in nifi.properties. > For example, a protectionScheme of `aes/gcm/256` would map to the > AESSensitivePropertiesProvider. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8611) GCP BigQuery processors support using designate project resource for ingestion
[
https://issues.apache.org/jira/browse/NIFI-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362030#comment-17362030
]
Joe Witt commented on NIFI-8611:
i'll review/merge. Just kicked off the checks in github
> GCP BigQuery processors support using designate project resource for ingestion
> --
>
> Key: NIFI-8611
> URL: https://issues.apache.org/jira/browse/NIFI-8611
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Chih Han Yu
>Assignee: Joe Witt
>Priority: Major
> Labels: GCP, bigquery
> Fix For: 1.14.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For now, *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor
> can only assign a single project id for consuming resources and do ingestion.
> But in some business cases, the project providing resources and the project
> which be inserted are not always the same.
> src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java
>
> {code:java}
> ..
> public static final PropertyDescriptor PROJECT_ID = new PropertyDescriptor
> .Builder().name("gcp-project-id")
> .displayName("Project ID")
> .description("Google Cloud Project ID")
> .required(false)
>
> .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
> .build();
> ..{code}
>
> We've test a solution which is workable, which is, adding another property
> *DESIGNATE_PROJECT_ID* in *AbstractBigQueryProcessor*, it'll only impact
> *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor.
> If user provides designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *DESIGNATE_PROJECT_ID* (defined in
> AbstractBigQueryProcessor).
> If user does {color:#ff}not{color} provide designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *PROJECT_ID* (defined in AbstractGCPProcessor).
> Since we already implemented this solution in production environment, I'll
> submit a PR later for this improvement.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-8611) GCP BigQuery processors support using designate project resource for ingestion
[
https://issues.apache.org/jira/browse/NIFI-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt reassigned NIFI-8611:
--
Assignee: Joe Witt
> GCP BigQuery processors support using designate project resource for ingestion
> --
>
> Key: NIFI-8611
> URL: https://issues.apache.org/jira/browse/NIFI-8611
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Chih Han Yu
>Assignee: Joe Witt
>Priority: Major
> Labels: GCP, bigquery
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For now, *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor
> can only assign a single project id for consuming resources and do ingestion.
> But in some business cases, the project providing resources and the project
> which be inserted are not always the same.
> src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java
>
> {code:java}
> ..
> public static final PropertyDescriptor PROJECT_ID = new PropertyDescriptor
> .Builder().name("gcp-project-id")
> .displayName("Project ID")
> .description("Google Cloud Project ID")
> .required(false)
>
> .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
> .build();
> ..{code}
>
> We've test a solution which is workable, which is, adding another property
> *DESIGNATE_PROJECT_ID* in *AbstractBigQueryProcessor*, it'll only impact
> *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor.
> If user provides designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *DESIGNATE_PROJECT_ID* (defined in
> AbstractBigQueryProcessor).
> If user does {color:#ff}not{color} provide designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *PROJECT_ID* (defined in AbstractGCPProcessor).
> Since we already implemented this solution in production environment, I'll
> submit a PR later for this improvement.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8611) GCP BigQuery processors support using designate project resource for ingestion
[
https://issues.apache.org/jira/browse/NIFI-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8611:
---
Fix Version/s: 1.14.0
> GCP BigQuery processors support using designate project resource for ingestion
> --
>
> Key: NIFI-8611
> URL: https://issues.apache.org/jira/browse/NIFI-8611
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Chih Han Yu
>Assignee: Joe Witt
>Priority: Major
> Labels: GCP, bigquery
> Fix For: 1.14.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For now, *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor
> can only assign a single project id for consuming resources and do ingestion.
> But in some business cases, the project providing resources and the project
> which be inserted are not always the same.
> src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java
>
> {code:java}
> ..
> public static final PropertyDescriptor PROJECT_ID = new PropertyDescriptor
> .Builder().name("gcp-project-id")
> .displayName("Project ID")
> .description("Google Cloud Project ID")
> .required(false)
>
> .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
> .build();
> ..{code}
>
> We've test a solution which is workable, which is, adding another property
> *DESIGNATE_PROJECT_ID* in *AbstractBigQueryProcessor*, it'll only impact
> *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor.
> If user provides designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *DESIGNATE_PROJECT_ID* (defined in
> AbstractBigQueryProcessor).
> If user does {color:#ff}not{color} provide designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *PROJECT_ID* (defined in AbstractGCPProcessor).
> Since we already implemented this solution in production environment, I'll
> submit a PR later for this improvement.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8560) 0.6.0 Release Notes
[ https://issues.apache.org/jira/browse/NIFI-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-8560: --- Fix Version/s: (was: 1.14.0) > 0.6.0 Release Notes > --- > > Key: NIFI-8560 > URL: https://issues.apache.org/jira/browse/NIFI-8560 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 0.6.0 >Reporter: Nathan Gough >Priority: Major > Labels: release, releasenotes > > This issue is tracking what work needs to be done for release notes to > release 0.6.0. > * Include dependency version changes > * Notify users that the OkHTTP upgrade will now require certificates to > include a DNS SAN entry, as recommended by > [https://tools.ietf.org/html/rfc6125]. Users will find that certificates > without a valid SAN entry in certificates used with MiNiFi will result in > failed TLS negotiation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-8344) Allow TailFile to continue tailing a file for some time after it has been rolled over
[ https://issues.apache.org/jira/browse/NIFI-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-8344. Resolution: Fixed > Allow TailFile to continue tailing a file for some time after it has been > rolled over > - > > Key: NIFI-8344 > URL: https://issues.apache.org/jira/browse/NIFI-8344 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > TailFile makes the assumption that once a file has been rolled over, it will > never be appended to. If the file's Last Modified timestamp changes, the > processor assumes that it's a new file and imports the entire contents of the > file again. > However, one practice that I've encountered is that users have a syslog > server that rotates periodically. To rotate, they rename the existing file, > and then restart the server. When that happens, the server will flush out any > data that it has buffered to the file that was just rolled over, and then > begin writing to the new file. > This results in the TailFile processor ingesting the entire file that has > been rolled over. Because we can't keep state about every file that is rolled > over, we should introduce a property that allows the user to indicate that > upon rollover they want to continue tailing that rolled over file until it is > no longer being written to, and then begin tailing the new file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8611) GCP BigQuery processors support using designate project resource for ingestion
[
https://issues.apache.org/jira/browse/NIFI-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8611:
---
Fix Version/s: (was: 1.14.0)
> GCP BigQuery processors support using designate project resource for ingestion
> --
>
> Key: NIFI-8611
> URL: https://issues.apache.org/jira/browse/NIFI-8611
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Chih Han Yu
>Priority: Major
> Labels: GCP, bigquery
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For now, *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor
> can only assign a single project id for consuming resources and do ingestion.
> But in some business cases, the project providing resources and the project
> which be inserted are not always the same.
> src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java
>
> {code:java}
> ..
> public static final PropertyDescriptor PROJECT_ID = new PropertyDescriptor
> .Builder().name("gcp-project-id")
> .displayName("Project ID")
> .description("Google Cloud Project ID")
> .required(false)
>
> .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
> .build();
> ..{code}
>
> We've test a solution which is workable, which is, adding another property
> *DESIGNATE_PROJECT_ID* in *AbstractBigQueryProcessor*, it'll only impact
> *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor.
> If user provides designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *DESIGNATE_PROJECT_ID* (defined in
> AbstractBigQueryProcessor).
> If user does {color:#ff}not{color} provide designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *PROJECT_ID* (defined in AbstractGCPProcessor).
> Since we already implemented this solution in production environment, I'll
> submit a PR later for this improvement.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-8220) Establish a secure by default configuration for NiFi
[ https://issues.apache.org/jira/browse/NIFI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-8220. Resolution: Fixed > Establish a secure by default configuration for NiFi > > > Key: NIFI-8220 > URL: https://issues.apache.org/jira/browse/NIFI-8220 > Project: Apache NiFi > Issue Type: Epic > Components: Tools and Build >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Blocker > Fix For: 1.14.0 > > > Inspired by this tweet > https://twitter.com/_escctrl_/status/1359280656174510081?s=21 and the > resulting discussion here > https://lists.apache.org/thread.html/rc590f21807192a0dce18293c2d5b47392a6fd8a1ef26d77fbd6ee695%40%3Cdev.nifi.apache.org%3E > It is time to change our config model. It was also setup to be easy to use. > We've seen these silly setups on the Internet before but has gotten > ridiculous. We need to take action. > Will create a set of one or more JIRAs to roughly do the following. > 1. Disable HTTP by default. If a user wants to enable to it for whatever > reason then also make them enable a new property which says something to the > effect of 'allow completely non secure access to the entire nifi instance - > not recommended' > 2. Enable HTTPS with one way authentication by default which would be the > client authenticating the server whereby the server has a server cert. We > could either make that cert a self-signed (and thus not trusted by client's > by default) cert or give a way for the user to run through command line > process to make a legit cert. > 3. If not already configured with an authorization provider supply and out of > the box provider which supports only a single auto generated at first startup > user/password enabling access to the NiFi system. > 4. Disable all restricted processors by default. Require the user to > explicitly enable them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8573) Migrate Docker image names to mirror those of the public repos
[ https://issues.apache.org/jira/browse/NIFI-8573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-8573: --- Fix Version/s: (was: 1.14.0) > Migrate Docker image names to mirror those of the public repos > -- > > Key: NIFI-8573 > URL: https://issues.apache.org/jira/browse/NIFI-8573 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > For consistency, it would be helpful to have the docker images reflect the > image tags used in Docker Hub. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8557) Expose bootstrap properties in the ConfigTransformer
[
https://issues.apache.org/jira/browse/NIFI-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8557:
---
Fix Version/s: (was: 1.14.0)
> Expose bootstrap properties in the ConfigTransformer
>
>
> Key: NIFI-8557
> URL: https://issues.apache.org/jira/browse/NIFI-8557
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Joe Percivall
>Assignee: Joe Percivall
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The ConfigTransformer takes in the config.yml and creates the nifi.properties
> and flow.xml. In order to better support customizations on a per MiNiFi
> instance for things that aren't able to reference EL, we could expose the
> properties listed in the bootstrap.conf.
> As an example, the bootstrap conf could have properties identifying the S2S
> URL and port UUID to use. Then when MiNiFi pulls down the new config.yml it
> would translate the keys to their proper values as identified in the
> bootstrap.conf.
> The main unknown is what the "escape" identifiers would be. In EL it is "${
> . }" (not sure why Jira is formatting this with new lines). This would
> need to be specific enough that it doesn't collide with anything that'd be in
> the config.yml.
> Much further down the line, this could eventually evolve to expose ENV
> variables, key/values stored in a file, and maybe even basic functions as
> needed. Essentially a basic version of EL but I hesitate to call it that b/c
> I don't want users to expect all of that functionality. This should really be
> for things that can't be done via EL.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8555) C2 Server Error - Too Many Open Files
[ https://issues.apache.org/jira/browse/NIFI-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-8555: --- Fix Version/s: (was: 1.14.0) > C2 Server Error - Too Many Open Files > - > > Key: NIFI-8555 > URL: https://issues.apache.org/jira/browse/NIFI-8555 > Project: Apache NiFi > Issue Type: Bug > Components: MiNiFi >Affects Versions: 0.5.0 > Environment: OS: Amazon Linux 2 (4.14.77-86.82.amzn2.x86_64) > Java: 1.8.0_191-b12 > ulimits: > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 31769 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 5 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 1 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited >Reporter: Michael >Priority: Blocker > > After several days of running C2 the errors are being logged about having > "Too Many Open Files". At this point, C2 is not responsive to clients trying > to check for new configurations. > _33e79e: Too many open files_ > _Apr 29 14:33:22 ec2.internal c2.sh[20851]: Caused by: > java.nio.file.FileSystemException: > /opt/minifi-c2/./files/5233730D-AC58-4853-9C0E-273D0E_ > > Running _lsof_ shows an increasingly larger number of open files. Our MiNiFi > instances are checking for new files every few seconds (probably too short), > but it seems like maybe some file handles are not being cleaned up. > Restarting C2 clears up this condition and C2 becomes responsive again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8556) minifi-docker module should depend on minifi-assembly
[ https://issues.apache.org/jira/browse/NIFI-8556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-8556: --- Fix Version/s: (was: 1.14.0) > minifi-docker module should depend on minifi-assembly > - > > Key: NIFI-8556 > URL: https://issues.apache.org/jira/browse/NIFI-8556 > Project: Apache NiFi > Issue Type: Bug > Components: MiNiFi >Reporter: Aldrin Piri >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Currently the docker module does not depend on the assembly. This means that > for multhreaded builds there is a potential that the docker build will make > use of what is a stale assembly. Or, for a clean state, will not be able to > build when activated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8689:
---
Fix Version/s: 1.14.0
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362014#comment-17362014
]
ASF subversion and git services commented on NIFI-8689:
---
Commit 64f600d0ce28be3856c1abdf947d05058ea548a7 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=64f600d ]
NIFI-8689: This closes #5150. Avoid flushing the socket buffer unnecessarily
when sending a series of FlowFiles via site-to-site
Signed-off-by: Joe Witt
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8689:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362010#comment-17362010
]
Joe Witt commented on NIFI-8689:
https://github.com/apache/nifi/pull/5150
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8633) Content Repository can be improved to make fewer disks accesses on read
[
https://issues.apache.org/jira/browse/NIFI-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8633:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Content Repository can be improved to make fewer disks accesses on read
> ---
>
> Key: NIFI-8633
> URL: https://issues.apache.org/jira/browse/NIFI-8633
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When {{FileSystemRepository.read(ContentClaim)}} or
> {{FileSystemRepository.read(ResourceClaim)}} is called, the repository
> determines the file path for the claim via {{getPath(claim, true);}} where
> the true indicates that we should verify that the file exists.
> This is done so that if we were to pass in a ContentClaim that does not
> exist, we throw a more meaningful ContentNotFoundException instead of just
> letting a FileNotFoundException fly.
> However, this call to {{Files.exists(Path)}} is fairly expensive, as it's a
> disk access. For a flow that uses a lot of smaller files, this can be
> extremely expensive.
> We can improve this by removing the call to {{Files.exists}} all together.
> Instead, just blindly create the {{FileInputStream}} in a try/catch block and
> catch FileNotFoundException, and then wrap that in a
> {{ContentNotFoundException}}. This results in the same API and the same
> contracts as before but avoids the overhead of additional disk accesses/seeks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8633) Content Repository can be improved to make fewer disks accesses on read
[
https://issues.apache.org/jira/browse/NIFI-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362009#comment-17362009
]
ASF subversion and git services commented on NIFI-8633:
---
Commit 172afac6ab32350caf6f882e6e7c18e86a5c8c82 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=172afac ]
NIFI-8633: This closes #5104. When reading a Content/Resource Claim from
FileSystemRepository, avoid the unnecessary Files.exists call and instead just
create a FileInputStream, catching FileNotFoundException
Signed-off-by: Joe Witt
> Content Repository can be improved to make fewer disks accesses on read
> ---
>
> Key: NIFI-8633
> URL: https://issues.apache.org/jira/browse/NIFI-8633
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When {{FileSystemRepository.read(ContentClaim)}} or
> {{FileSystemRepository.read(ResourceClaim)}} is called, the repository
> determines the file path for the claim via {{getPath(claim, true);}} where
> the true indicates that we should verify that the file exists.
> This is done so that if we were to pass in a ContentClaim that does not
> exist, we throw a more meaningful ContentNotFoundException instead of just
> letting a FileNotFoundException fly.
> However, this call to {{Files.exists(Path)}} is fairly expensive, as it's a
> disk access. For a flow that uses a lot of smaller files, this can be
> extremely expensive.
> We can improve this by removing the call to {{Files.exists}} all together.
> Instead, just blindly create the {{FileInputStream}} in a try/catch block and
> catch FileNotFoundException, and then wrap that in a
> {{ContentNotFoundException}}. This results in the same API and the same
> contracts as before but avoids the overhead of additional disk accesses/seeks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8442) Can not write Timestamp, Time and Date in BigQuery
[ https://issues.apache.org/jira/browse/NIFI-8442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-8442: --- Fix Version/s: (was: 1.13.2) > Can not write Timestamp, Time and Date in BigQuery > -- > > Key: NIFI-8442 > URL: https://issues.apache.org/jira/browse/NIFI-8442 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.1, 1.13.2 >Reporter: Julien G. >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > When I have a Record with Timestamp, Time or Date, the Record can not be > inserted in the BigQuery Table. It's because the Java types > Java.sql.Timestamp, Java.sql.Date and Java.sql.Time are not managed by the > GCP library, so the field should be convert to String. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8690) NarBundleExtractor fails for latest extension docs
[ https://issues.apache.org/jira/browse/NIFI-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-8690: -- Status: Patch Available (was: Open) > NarBundleExtractor fails for latest extension docs > -- > > Key: NIFI-8690 > URL: https://issues.apache.org/jira/browse/NIFI-8690 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.13.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > > The NarBundleExtractor from NiFi Registry bundle utils uses a Jackson XML > deserializer to unmarshall the extension docs from inside a NAR. Currently > the object model hasn't been kept fully up to date with additions to the > extensions docs, so it will fail on NARs from 1.13.2 because of dependent > properties. Although we should update the object model at some point, we > should also configure the marshaller to ignore unknown properties. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8690) NarBundleExtractor fails for latest extension docs
Bryan Bende created NIFI-8690: - Summary: NarBundleExtractor fails for latest extension docs Key: NIFI-8690 URL: https://issues.apache.org/jira/browse/NIFI-8690 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Affects Versions: 1.13.2 Reporter: Bryan Bende Assignee: Bryan Bende The NarBundleExtractor from NiFi Registry bundle utils uses a Jackson XML deserializer to unmarshall the extension docs from inside a NAR. Currently the object model hasn't been kept fully up to date with additions to the extensions docs, so it will fail on NARs from 1.13.2 because of dependent properties. Although we should update the object model at some point, we should also configure the marshaller to ignore unknown properties. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361974#comment-17361974 ] ASF subversion and git services commented on NIFI-8625: --- Commit 3e774bc5beadd62a646794cc659db9902c5ab1fa in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3e774bc ] NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues (#5116) NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Assignee: Matt Burgess >Priority: Critical > Fix For: 1.14.0 > > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8625: - Fix Version/s: 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Assignee: Matt Burgess >Priority: Critical > Fix For: 1.14.0 > > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361975#comment-17361975 ] ASF subversion and git services commented on NIFI-8625: --- Commit 3e774bc5beadd62a646794cc659db9902c5ab1fa in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3e774bc ] NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues (#5116) NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Assignee: Matt Burgess >Priority: Critical > Fix For: 1.14.0 > > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8516) Set Default HTTPS and Single User Authentication
[ https://issues.apache.org/jira/browse/NIFI-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8516: - Fix Version/s: 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Set Default HTTPS and Single User Authentication > > > Key: NIFI-8516 > URL: https://issues.apache.org/jira/browse/NIFI-8516 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: HTTPS, security > Fix For: 1.14.0 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Building on the implementation of single user authentication in NIFI-8363 and > self-signed certificate generation in NIFI-8403, the default configuration > should be updated to enable these features. > This change should disable default access over unencrypted and > unauthenticated channels. The project README should be updated to reflect the > default behavior with instructions on retrieving the generated username and > password from application logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8516) Set Default HTTPS and Single User Authentication
[ https://issues.apache.org/jira/browse/NIFI-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361942#comment-17361942 ] ASF subversion and git services commented on NIFI-8516: --- Commit db40989b4833cb4281b7082712b43be78998d071 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=db40989 ] NIFI-8516 Enabled HTTPS and Single User Authentication in default configuration - Set default HTTPS Port to 9443 - Set default authorizer to single-user-authorizer - Set default login-identity-provider to single-user-provider - Updated README.md with authentication instructions using generated credentials - Updated default URL and port information in Administration and User Guides - Updated Getting Started Guide with authentication and URL changes - Updated Docker images to set HTTPS as default configuration - Updated default HTTPS port to 8443 - Set Cluster Protocol secure property in Docker start scripts - Added set-single-user-credentials command - Refactored shared classes to nifi-single-user-utils - Updated Getting Started documentation and logging - Updated documentation and TLS Toolkit default ports - Updated Toolkit Guide and Administration Guide - Updated README.md with HTTPS links > Set Default HTTPS and Single User Authentication > > > Key: NIFI-8516 > URL: https://issues.apache.org/jira/browse/NIFI-8516 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: HTTPS, security > Time Spent: 5h 50m > Remaining Estimate: 0h > > Building on the implementation of single user authentication in NIFI-8363 and > self-signed certificate generation in NIFI-8403, the default configuration > should be updated to enable these features. > This change should disable default access over unencrypted and > unauthenticated channels. The project README should be updated to reflect the > default behavior with instructions on retrieving the generated username and > password from application logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8689:
-
Status: Patch Available (was: Open)
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361912#comment-17361912
]
Mark Payne commented on NIFI-8689:
--
I ran some performance tests. When sending from my laptop back to itself, with
the following flow:
GenerateFlowFile (100 bytes) -> RPG (back to self)
Input Port -> UpdateAttribute
The throughput that I got when using 3 threads fro RPG and 3 threads for
InputPort was 3.3 MM FlowFiles/5 minutes
With 1 thread, I got 3.4 MM FlowFiles/5 mins
I then moved the InputPort -> UpdateAttribute flow to a different NiFi cluster,
where there was a high latency link (about 65 to 80 ms RTT from ping).
Using 3 threads, I got 805,000 FlowFiles/5 mins
This is summarized here:
||Number of Threads||Destination||Number of FlowFiles Sent in 5 minutes||
|1|Local|3.3 million|
|3|Local|3.4 million|
|3|Remote|805,000|
Then I updated the code to avoid those extraneous buffer flushes and saw the
following performance:
||Number of Threads||Destination||Number of FlowFiles Sent in 5
minutes||Percent Improvement||
|1|Local|4.64 million|40%|
|3|Local|5.46 million|65%|
|3|Remote|2.58 million|320%|
The performance was similar using the HTTP-based protocol.
> Site-to-Site client is constantly flushing the socket's OutputStream
>
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
>
>
>
> {code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*
> *Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream
Mark Payne created NIFI-8689:
Summary: Site-to-Site client is constantly flushing the socket's
OutputStream
Key: NIFI-8689
URL: https://issues.apache.org/jira/browse/NIFI-8689
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
When a RemoteProcessGroup is sending data to another NiFi instance, the
protocol should establish a transaction and then send a sequence of FlowFiles
following a pattern along the lines of:
{code:java}
{code}
However, currently, the protocol is flushing the Socket's output buffer each
that that it indicates that a FlowFile follows, and again after each FlowFile.
So it's more like:
{code:java}
*Flush Buffer*
*Flush Buffer*
*Flush Buffer*
*Flush Buffer*
*Flush Buffer*
*Flush Buffer*{code}
As a result, when sending a large number of smaller FlowFiles, we end up
constantly flushing data to the socket, which results in dramatically worse
performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8686) update administration guide with time_wait setting for kernel 3
[ https://issues.apache.org/jira/browse/NIFI-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Atwood updated NIFI-8686: Labels: pull-request-available (was: ) Status: Patch Available (was: In Progress) https://github.com/apache/nifi/pull/5149 > update administration guide with time_wait setting for kernel 3 > --- > > Key: NIFI-8686 > URL: https://issues.apache.org/jira/browse/NIFI-8686 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew Atwood >Assignee: Andrew Atwood >Priority: Trivial > Labels: pull-request-available > > the administration guide has below line to set how long sockets stay in a > TIMED_WAIT state when closed: > sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" > That is for kernel 2.6. For kernel 3, the property is: > net.netfilter.nf_conntrack_tcp_timeout_time_wait > I believe the value should still be "1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8688) Apache Nifi Cli properties file password encryption
Santhosh K created NIFI-8688: Summary: Apache Nifi Cli properties file password encryption Key: NIFI-8688 URL: https://issues.apache.org/jira/browse/NIFI-8688 Project: Apache NiFi Issue Type: New Feature Components: Tools and Build Affects Versions: 1.13.2 Environment: QA and Prod Reporter: Santhosh K Currently Nifi Cli.sh toolkit provides uses properties file to read keystore and password for client cert, but the properties file has password in clear text to read truststore and keystore. This password may need to be encrpyted or least encoded that cli.sh in toolkit can read part of loading keystore/truststore while connecting to Nifi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1591) Upgrade cxxopts and download it during build
[ https://issues.apache.org/jira/browse/MINIFICPP-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1591: Description: Because of this issue: https://github.com/jarro2783/cxxopts/pull/172 > Upgrade cxxopts and download it during build > > > Key: MINIFICPP-1591 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1591 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.10.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.11.0 > > > Because of this issue: https://github.com/jarro2783/cxxopts/pull/172 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Created] (NIFI-8687) Apache Nifi Cli properties file password encryption
Santhosh K created NIFI-8687: Summary: Apache Nifi Cli properties file password encryption Key: NIFI-8687 URL: https://issues.apache.org/jira/browse/NIFI-8687 Project: Apache NiFi Issue Type: New Feature Components: Tools and Build Affects Versions: 1.13.2 Environment: QA Reporter: Santhosh K NiFi cli currently only interacts using the client cert for athentication, is there a way to use user ID password to be used for cli.sh to connect to Nifi secure cluster? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8687) Apache Nifi Cli to authenticate with user credentials insted of client cert
[ https://issues.apache.org/jira/browse/NIFI-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santhosh K updated NIFI-8687: - Summary: Apache Nifi Cli to authenticate with user credentials insted of client cert (was: Apache Nifi Cli properties file password encryption) > Apache Nifi Cli to authenticate with user credentials insted of client cert > --- > > Key: NIFI-8687 > URL: https://issues.apache.org/jira/browse/NIFI-8687 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.13.2 > Environment: QA >Reporter: Santhosh K >Priority: Major > > NiFi cli currently only interacts using the client cert for athentication, is > there a way to use user ID password to be used for cli.sh to connect to Nifi > secure cluster? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Labels: InvokeScriptedProcessor property sensitive (was: InvokeScriptedProcessor) > sensitive property not working for InvokeScriptedProcessor > -- > > Key: NIFI-8684 > URL: https://issues.apache.org/jira/browse/NIFI-8684 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Jul Tomten >Priority: Major > Labels: InvokeScriptedProcessor, property, sensitive > > I use InvokeScriptedProcessor > I'm trying to read a sensitive property from the process context > > before restarting NiFi it was working fine > after restarting NiFi - NiFi fails to startup with the error below > see https://issues.apache.org/jira/browse/NIFI-7012 > > > 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.controller.serialization.FlowSynchronizationException: > java.lang.IllegalArgumentException: The property 'Password3' cannot reference > Parameter 'password3' because Sensitive Parameters may only be referenced by > Sensitive Properties. > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.server.Server.start(Server.java:423) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at >
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Created] (MINIFICPP-1591) Upgrade cxxopts and download it during build
Marton Szasz created MINIFICPP-1591: --- Summary: Upgrade cxxopts and download it during build Key: MINIFICPP-1591 URL: https://issues.apache.org/jira/browse/MINIFICPP-1591 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.10.0 Reporter: Marton Szasz Assignee: Marton Szasz Fix For: 0.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8686) update administration guide with time_wait setting for kernel 3
[ https://issues.apache.org/jira/browse/NIFI-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Atwood reassigned NIFI-8686: --- Assignee: Andrew Atwood > update administration guide with time_wait setting for kernel 3 > --- > > Key: NIFI-8686 > URL: https://issues.apache.org/jira/browse/NIFI-8686 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew Atwood >Assignee: Andrew Atwood >Priority: Trivial > > the administration guide has below line to set how long sockets stay in a > TIMED_WAIT state when closed: > sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" > That is for kernel 2.6. For kernel 3, the property is: > net.netfilter.nf_conntrack_tcp_timeout_time_wait > I believe the value should still be "1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-4613) Provenance search results are no longer display total numbered count
[ https://issues.apache.org/jira/browse/NIFI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361845#comment-17361845 ] Andrew Atwood commented on NIFI-4613: - I don't know why it isn't linked but this is PR: https://github.com/apache/nifi/pull/5148 > Provenance search results are no longer display total numbered count > > > Key: NIFI-4613 > URL: https://issues.apache.org/jira/browse/NIFI-4613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Juan C. Sequeiros >Assignee: Andrew Atwood >Priority: Minor > Labels: pull-request-available > > At least until 0.7 when doing a provenance search and it returns more than 1K > results, on the results page it would say displaying 1,000 of < total number > of results > > Ex: 1,000 of 500,000 > Now it returns if more than 1,000: > Displaying 1,000 of 1,000 + > We lose the quick indication of how many results it found. > It loses ability to quickly compare searches. > I could certainly narrow my searches but before the ability to at least know > a numbered result was a quick way to troubleshoot or get stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-4613) Provenance search results are no longer display total numbered count
[ https://issues.apache.org/jira/browse/NIFI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Atwood updated NIFI-4613: Labels: pull-request-available (was: ) Status: Patch Available (was: In Progress) > Provenance search results are no longer display total numbered count > > > Key: NIFI-4613 > URL: https://issues.apache.org/jira/browse/NIFI-4613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Juan C. Sequeiros >Assignee: Andrew Atwood >Priority: Minor > Labels: pull-request-available > > At least until 0.7 when doing a provenance search and it returns more than 1K > results, on the results page it would say displaying 1,000 of < total number > of results > > Ex: 1,000 of 500,000 > Now it returns if more than 1,000: > Displaying 1,000 of 1,000 + > We lose the quick indication of how many results it found. > It loses ability to quickly compare searches. > I could certainly narrow my searches but before the ability to at least know > a numbered result was a quick way to troubleshoot or get stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8686) update administration guide with time_wait setting for kernel 3
Andrew Atwood created NIFI-8686: --- Summary: update administration guide with time_wait setting for kernel 3 Key: NIFI-8686 URL: https://issues.apache.org/jira/browse/NIFI-8686 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Andrew Atwood the administration guide has below line to set how long sockets stay in a TIMED_WAIT state when closed: sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" That is for kernel 2.6. For kernel 3, the property is: net.netfilter.nf_conntrack_tcp_timeout_time_wait I believe the value should still be "1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1550) CMake Error at libminifi/CMakeLists.txt: Cannot find source file: src/agent/agent_version.cpp (FreeBSD 13.0 x86_64)
[ https://issues.apache.org/jira/browse/MINIFICPP-1550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1550: Affects Version/s: (was: 1.0.0) 0.9.0 > CMake Error at libminifi/CMakeLists.txt: Cannot find source file: > src/agent/agent_version.cpp (FreeBSD 13.0 x86_64) > - > > Key: MINIFICPP-1550 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1550 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.9.0 > Environment: $ uname -a > FreeBSD gollvm.build.server 13.0-CURRENT FreeBSD 13.0-CURRENT #0 > b9403d7aae8-c254071(main): Thu Oct 29 08:06:03 UTC 2020 > r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > $ cmake --version > cmake version 3.18.4 > $ clang --version > clang version 11.0.0 > Target: x86_64-unknown-freebsd11.4 > Thread model: posix > $ ninja --version > 1.10.1 > $bison --version > bison (GNU Bison) 3.6.4 > $ flex --version > flex 2.6.4 >Reporter: Ivan Serdyuk >Priority: Blocker > Attachments: cmake_trace.log, cmake_trace_expand.log > > > $ cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -G Ninja .. > CMake Deprecation Warning at CMakeLists.txt:21 (cmake_policy): > The OLD behavior for policy CMP0065 will be removed from a future version > of CMake. > The cmake-policies(7) manual explains that the OLD behaviors of all > policies are deprecated and that a policy should be set to OLD only under > specific short-term circumstances. Projects should be ported to the NEW > behavior and not rely on setting a policy to OLD. > -- The C compiler identification is Clang 11.0.0 > -- The CXX compiler identification is Clang 11.0.0 > -- Detecting C compiler ABI info > -- Detecting C compiler ABI info - done > -- Check for working C compiler: /usr/bin/clang - skipped > -- Detecting C compile features > -- Detecting C compile features - done > -- Detecting CXX compiler ABI info > -- Detecting CXX compiler ABI info - done > -- Check for working CXX compiler: /usr/bin/clang++ - skipped > -- Detecting CXX compile features > -- Detecting CXX compile features - done > -- Looking for execinfo.h > -- Looking for execinfo.h - not found > -- Performing Test COMPILER_SUPPORTS_CXX11 > -- Performing Test COMPILER_SUPPORTS_CXX11 - Success > -- Performing Test COMPILER_SUPPORTS_CXX0X > -- Performing Test COMPILER_SUPPORTS_CXX0X - Success > -- Looking for pthread.h > -- Looking for pthread.h - found > -- Performing Test CMAKE_HAVE_LIBC_PTHREAD > -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed > -- Check if compiler accepts -pthread > -- Check if compiler accepts -pthread - yes > -- Found Threads: TRUE > CMAKE_THREAD_LIBS_INIT is -pthread > THREADS_HAVE_PTHREAD_ARG is TRUE > -- Found Patch: /usr/bin/patch > Using bundled ossp-uuid > Using bundled LibreSSL > Using bundled libsodium > Using bundled zlib > CMake Warning (dev) at libminifi/CMakeLists.txt:22 (project): > Policy CMP0048 is not set: project() command manages VERSION variables. > Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy > command to set the policy and suppress this warning. > The following variable(s) would be set to empty: > PROJECT_VERSION_MAJOR > PROJECT_VERSION_MINOR > PROJECT_VERSION_PATCH > This warning is for project developers. Use -Wno-dev to suppress it. > minifi-http-curl will depend on curl-external > -- Found BISON: /usr/local/bin/bison (found version "3.6.4") > -- Found FLEX: /usr/bin/flex (found version "2.6.4") > # date: USE_SYSTEM_TZ_DB ON > # date: USE_TZ_DB_IN_DOT OFF > # date: BUILD_SHARED_LIBS OFF > # date: ENABLE_DATE_TESTING OFF > minifi-expression-language-extensions will depend on curl-external > Using bundled civetweb > Using bundled RocksDB > Using bundled liblzma > Using bundled bzip2 > -- Performing Test COMPILER_SUPPORTS_CXX14 > -- Performing Test COMPILER_SUPPORTS_CXX14 - Success > -- Performing Test COMPILER_SUPPORTS_CXX1Y > -- Performing Test COMPILER_SUPPORTS_CXX1Y - Success > -- Found PythonLibs: /usr/local/lib/libpython3.7m.so (found suitable version > "3.7.9", minimum required is "3.5") > CMake Deprecation Warning at main/CMakeLists.txt:23 (CMAKE_POLICY): > The OLD behavior for policy CMP0048 will be removed from a future version > of CMake. > The cmake-policies(7) manual explains that the OLD behaviors of all > policies are deprecated and that a policy should be set to OLD only under > specific short-term circumstances. Projects should be ported to the NEW > behavior and not rely on setting a policy to OLD. > Linking MiNiFiMain against minifi-standard-processors > Linking MiNiFiMain against minifi-http-curl > Linking MiNiFiMain against
[jira] [Created] (NIFI-8685) [NIFI-SITE] nifi site list of committers and pmc is out of date
Otto Fowler created NIFI-8685: - Summary: [NIFI-SITE] nifi site list of committers and pmc is out of date Key: NIFI-8685 URL: https://issues.apache.org/jira/browse/NIFI-8685 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Otto Fowler Looks like it hasn't been updated in a while. We need the current list and to update it -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Commented] (NIFI-7012) Using sensitive parameter in sensitive property of InvokeScriptedProcessor causes Jetty shutdown on NiFi restart
[
https://issues.apache.org/jira/browse/NIFI-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361695#comment-17361695
]
Jul Tomten commented on NIFI-7012:
--
I have the same issue,
https://issues.apache.org/jira/browse/NIFI-8684
sorry for creating a duplicate
> Using sensitive parameter in sensitive property of InvokeScriptedProcessor
> causes Jetty shutdown on NiFi restart
>
>
> Key: NIFI-7012
> URL: https://issues.apache.org/jira/browse/NIFI-7012
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.10.0
> Environment: OpenJDK 1.8.0_232 on Ubuntu 18.04.3 LTS
>Reporter: Dariusz Chmielewski
>Assignee: Andy LoPresto
>Priority: Major
> Labels: parameters, scripting, security
> Attachments: groovy_sample.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> To simulate this add a new InvokeScriptedProcessor to your flow with attached
> example basic Groovy script as the Script Body. This will create a "Password"
> sensitive property on the processor, then using UI enter sensitive parameter
> (ex: "#\{test_pass}") in the value of this property. At this point everything
> will work fine, no errors in UI or nifi-app.log during this processor
> execution, but when you restart NiFi you will notice the below warning in
> nifi-app.log. To get around this issue and have Jetty start again I had to
> manually edit flow.xml and set the parameter ("test_pass") sensitive value to
> false. Note that both the processor property and parameter appear as
> sensitive in UI (before making changes to flow.xml) and their values are
> encrypted in flow.xml.
> {code:java}
> 2020-01-10 15:37:28,492 INFO [main] org.eclipse.jetty.server.Server Started
> @29687ms
> 2020-01-10 15:37:28,493 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: The property 'Password' cannot reference
> Parameter 'test_pass' because Sensitive Parameters may only be referenced by
> Sensitive Properties.
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:510)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1368)
> at
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:88)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:812)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:557)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:959)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:924)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:365)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
> at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:406)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
> at
>
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Component/s: Extensions > sensitive property not working for InvokeScriptedProcessor > -- > > Key: NIFI-8684 > URL: https://issues.apache.org/jira/browse/NIFI-8684 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Jul Tomten >Priority: Major > > I use InvokeScriptedProcessor > I'm trying to read a sensitive property from the process context > > before restarting NiFi it was working fine > after restarting NiFi - NiFi fails to startup with the error below > > > 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.controller.serialization.FlowSynchronizationException: > java.lang.IllegalArgumentException: The property 'Password3' cannot reference > Parameter 'password3' because Sensitive Parameters may only be referenced by > Sensitive Properties. > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.server.Server.start(Server.java:423) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.server.Server.doStart(Server.java:387) > at >
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Labels: InvokeScriptedProcessor (was: ) > sensitive property not working for InvokeScriptedProcessor > -- > > Key: NIFI-8684 > URL: https://issues.apache.org/jira/browse/NIFI-8684 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.13.2 >Reporter: Jul Tomten >Priority: Major > Labels: InvokeScriptedProcessor > > I use InvokeScriptedProcessor > I'm trying to read a sensitive property from the process context > > before restarting NiFi it was working fine > after restarting NiFi - NiFi fails to startup with the error below > > > 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.controller.serialization.FlowSynchronizationException: > java.lang.IllegalArgumentException: The property 'Password3' cannot reference > Parameter 'password3' because Sensitive Parameters may only be referenced by > Sensitive Properties. > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.server.Server.start(Server.java:423) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.server.Server.doStart(Server.java:387) > at
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below see https://issues.apache.org/jira/browse/NIFI-7012 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Affects Version/s: 1.13.2 > sensitive property not working for InvokeScriptedProcessor > -- > > Key: NIFI-8684 > URL: https://issues.apache.org/jira/browse/NIFI-8684 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 >Reporter: Jul Tomten >Priority: Major > > I use InvokeScriptedProcessor > I'm trying to read a sensitive property from the process context > > before restarting NiFi it was working fine > after restarting NiFi - NiFi fails to startup with the error below > > > 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.controller.serialization.FlowSynchronizationException: > java.lang.IllegalArgumentException: The property 'Password3' cannot reference > Parameter 'password3' because Sensitive Parameters may only be referenced by > Sensitive Properties. > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.server.Server.start(Server.java:423) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.server.Server.doStart(Server.java:387) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361576#comment-17361576
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 12:13 PM:
---
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: testname' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "testname"
}
}'
{code}
This will force the {{ApplicationResource#generateId}} method's line *210* to
throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* UUID v3 value.
was (Author: aprakash):
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: testname' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "testname"
}
}'
{code}
This will force the {{ApplicationResource#generateId}} method's line *210* to
throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361576#comment-17361576
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 12:12 PM:
---
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: testname' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "testname"
}
}'
{code}
This will force the {{ApplicationResource#generateId}} method's line *210* to
throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
was (Author: aprakash):
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the {{ApplicationResource#generateId}} method's line *210* to
throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Commented] (NIFI-8642) Select the default Old Gen Memory Pool for Memory Reporting Task
[ https://issues.apache.org/jira/browse/NIFI-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361683#comment-17361683 ] ASF subversion and git services commented on NIFI-8642: --- Commit d7a8d275c96040627dd357fa76ba4a8276df8682 in nifi's branch refs/heads/main from Timea Barna [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d7a8d27 ] NIFI-8642 Select the default Old Gen Memory Pool for Memory Reporting Task This closes #5115. Signed-off-by: Tamas Palfy > Select the default Old Gen Memory Pool for Memory Reporting Task > > > Key: NIFI-8642 > URL: https://issues.apache.org/jira/browse/NIFI-8642 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > Add some code in a static method (which can then be called when building the > descriptor and setting its default value) that determines which memory pools > are available and then chooses one, based on a set of hardcoded values. I.e., > "G1 Old Gen", "PS Old Gen" or whatever is the name of the Old Generation > memory pool. Would need to find the list of possible values from somewhere. > At a minimum, we could at least use G1 Old Gen and the one for Java's default > GC algorithm. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jul Tomten updated NIFI-8684: - Description: I use InvokeScriptedProcessor I'm trying to read a sensitive property from the process context before restarting NiFi it was working fine after restarting NiFi - NiFi fails to startup with the error below 2021-06-11 11:22:09,673 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1057) at org.apache.nifi.NiFi.(NiFi.java:159) at org.apache.nifi.NiFi.(NiFi.java:71) at org.apache.nifi.NiFi.main(NiFi.java:303) Caused by: java.lang.IllegalArgumentException: The property 'Password3' cannot reference Parameter 'password3' because Sensitive Parameters may only be referenced by Sensitive Properties. at org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:313) at
[jira] [Created] (NIFI-8684) sensitive property not working for InvokeScriptedProcessor
Jul Tomten created NIFI-8684: Summary: sensitive property not working for InvokeScriptedProcessor Key: NIFI-8684 URL: https://issues.apache.org/jira/browse/NIFI-8684 Project: Apache NiFi Issue Type: Bug Reporter: Jul Tomten -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8683) SSLContextService should allow Expression Language to be used for TRUSTSTORE and KEYSTORE
[
https://issues.apache.org/jira/browse/NIFI-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-8683:
---
Assignee: Chris Sampson
> SSLContextService should allow Expression Language to be used for TRUSTSTORE
> and KEYSTORE
> -
>
> Key: NIFI-8683
> URL: https://issues.apache.org/jira/browse/NIFI-8683
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.13.2
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
>
> It would be handy (in clustered environments) for the {{SSLContextService}}
> to allow Expression Language to be used for specifying the TRUSTSTORE and
> KEYSTORE properties.
> This would allow users to use an expression like
> {quote}
> "/opt/nifi/nifi-current/conf/certs/${hostname(false)}.jks"
> {quote}
> to reference files that are unique to each host within the cluster (e.g. if
> using TLS protected communications for Site-To-Site Reporting from a
> cluster). Each file would still need to use the same password, but at least
> each host could have its own uniquely named certificate file (instead of
> having to create the same file on each host, which can lead to users
> incorrectly creating wildcard certificates for their clusters, which is
> discouraged).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-8683) SSLContextService should allow Expression Language to be used for TRUSTSTORE and KEYSTORE
Chris Sampson created NIFI-8683:
---
Summary: SSLContextService should allow Expression Language to be
used for TRUSTSTORE and KEYSTORE
Key: NIFI-8683
URL: https://issues.apache.org/jira/browse/NIFI-8683
Project: Apache NiFi
Issue Type: Improvement
Affects Versions: 1.13.2
Reporter: Chris Sampson
It would be handy (in clustered environments) for the {{SSLContextService}} to
allow Expression Language to be used for specifying the TRUSTSTORE and KEYSTORE
properties.
This would allow users to use an expression like
{quote}
"/opt/nifi/nifi-current/conf/certs/${hostname(false)}.jks"
{quote}
to reference files that are unique to each host within the cluster (e.g. if
using TLS protected communications for Site-To-Site Reporting from a cluster).
Each file would still need to use the same password, but at least each host
could have its own uniquely named certificate file (instead of having to create
the same file on each host, which can lead to users incorrectly creating
wildcard certificates for their clusters, which is discouraged).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361598#comment-17361598 ] Joseph Gresock commented on NIFI-8643: -- Good catch Jens, I've submitted a PR here: https://github.com/apache/nifi/pull/5146 > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock reassigned NIFI-8643: Assignee: Joseph Gresock > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8643: - Status: Patch Available (was: Open) > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Assignee: Joseph Gresock >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361576#comment-17361576
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:46 AM:
--
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the \{{ApplicationResource#generateId }}method's\{{ }}line
*210* to throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
was (Author: aprakash):
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the {{ApplicationResource#generateId }}method's{{ }}line *210*
to throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a VALID value_._
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361576#comment-17361576
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:46 AM:
--
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the {{ApplicationResource#generateId}} method's line *210* to
throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
was (Author: aprakash):
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the \{{ApplicationResource#generateId }}method's\{{ }}line
*210* to throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a *VALID* value.
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361576#comment-17361576
]
Arun Prakash commented on NIFI-8669:
One way to _hack_ this issue currently is using the
_X-Cluster-Id-Generation-Seed_ header key as follows:
{code:java}
curl --location --request POST
'http://localhost:9091/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'X-Cluster-Id-Generation-Seed: arun' \
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
This will force the {{ApplicationResource#generateId }}method's{{ }}line *210*
to throw an error.
{code:java}
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
{code}
which will then allow the fallback solution of generating the *UUID* with the
built in _nameUUIDFromBytes_ resulting in a VALID value_._
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361536#comment-17361536
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:17 AM:
--
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
A bit more background on when the issue arises.
I am using the nifi-api to create ProcessGroups. I use, the following REST POST
request:
{code:java}
curl --location --request POST
'http://localhost:9090/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
which returns
{code:json}
{
"revision": {
"version": 1,
"lastModifier": "anonymous"
},
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"uri":
"http://localhost:9090/nifi-api/process-groups/fa4829ef-0179-1000-5e44-7fc558cbb6d3;,
"position": {
"x": 0.0,
"y": 0.0
},
"permissions": {
"canRead": true,
"canWrite": true
},
"bulletins": [],
"component": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"parentGroupId": "58d527f9-0178-1000-21a9-cf75fa6606c0",
"position": {
"x": 0.0,
"y": 0.0
},
"name": "test",
"comments": "",
"variables": {},
"flowfileConcurrency": "UNBOUNDED",
"flowfileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"contents": {
"processGroups": [],
"remoteProcessGroups": [],
"processors": [],
"inputPorts": [],
"outputPorts": [],
"connections": [],
"labels": [],
"funnels": [],
"controllerServices": []
},
"inputPortCount": 0,
"outputPortCount": 0
},
"status": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"statsLastRefreshed": "08:55:04 UTC",
"aggregateSnapshot": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"flowFilesIn": 0,
"bytesIn": 0,
"input": "0 (0 bytes)",
"flowFilesQueued": 0,
"bytesQueued": 0,
"queued": "0 (0 bytes)",
"queuedCount": "0",
"queuedSize": "0 bytes",
"bytesRead": 0,
"read": "0 bytes",
"bytesWritten": 0,
"written": "0 bytes",
"flowFilesOut": 0,
"bytesOut": 0,
"output": "0 (0 bytes)",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "0 (0 bytes)",
"bytesReceived": 0,
"flowFilesReceived": 0,
"received": "0 (0 bytes)",
"bytesSent": 0,
"flowFilesSent": 0,
"sent": "0 (0 bytes)",
"activeThreadCount": 0,
"terminatedThreadCount": 0
}
},
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"inputPortCount": 0,
"outputPortCount": 0
}
{code}
Decoding using the [https://www.uuidtools.com/decode] we can see that the *id*
{{"id": "fa4829ef-0179-*1*000-*5*e44-7fc558cbb6d3"}}
has
{code:java}
Version 1 (time and node based)
Variant reserved (NCS backward compatible)
{code}
which makes it unfortunately *INVALID*, due to the use of reserved characters
for the _Variant_
Assuming that the *POST* request is handled by:
{code:java}
org.apache.nifi.web.api.ProcessGroupResource#createProcessGroup()
{code}
the *UUID* is generated in line *2036*
{code:java}
// set the processor id as appropriate
processGroup.setId(generateUuid());
{code}
which in turn is a call to the inherited method in the abstract class
{{ApplicationResource}}
{code:java}
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361536#comment-17361536
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:15 AM:
--
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
A bit more background on when the issue arises.
I am using the nifi-api to create ProcessGroups. I use, the following REST POST
request:
{code:java}
curl --location --request POST
'http://localhost:9090/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
which returns
{code:json}
{
"revision": {
"version": 1,
"lastModifier": "anonymous"
},
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"uri":
"http://localhost:9090/nifi-api/process-groups/fa4829ef-0179-1000-5e44-7fc558cbb6d3;,
"position": {
"x": 0.0,
"y": 0.0
},
"permissions": {
"canRead": true,
"canWrite": true
},
"bulletins": [],
"component": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"parentGroupId": "58d527f9-0178-1000-21a9-cf75fa6606c0",
"position": {
"x": 0.0,
"y": 0.0
},
"name": "test",
"comments": "",
"variables": {},
"flowfileConcurrency": "UNBOUNDED",
"flowfileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"contents": {
"processGroups": [],
"remoteProcessGroups": [],
"processors": [],
"inputPorts": [],
"outputPorts": [],
"connections": [],
"labels": [],
"funnels": [],
"controllerServices": []
},
"inputPortCount": 0,
"outputPortCount": 0
},
"status": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"statsLastRefreshed": "08:55:04 UTC",
"aggregateSnapshot": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"flowFilesIn": 0,
"bytesIn": 0,
"input": "0 (0 bytes)",
"flowFilesQueued": 0,
"bytesQueued": 0,
"queued": "0 (0 bytes)",
"queuedCount": "0",
"queuedSize": "0 bytes",
"bytesRead": 0,
"read": "0 bytes",
"bytesWritten": 0,
"written": "0 bytes",
"flowFilesOut": 0,
"bytesOut": 0,
"output": "0 (0 bytes)",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "0 (0 bytes)",
"bytesReceived": 0,
"flowFilesReceived": 0,
"received": "0 (0 bytes)",
"bytesSent": 0,
"flowFilesSent": 0,
"sent": "0 (0 bytes)",
"activeThreadCount": 0,
"terminatedThreadCount": 0
}
},
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"inputPortCount": 0,
"outputPortCount": 0
}
{code}
Decoding using the [https://www.uuidtools.com/decode] we can see that the *id*
{{"id": "fa4829ef-0179-*1*000-*5*e44-7fc558cbb6d3"}}
has
{code:java}
Version 1 (time and node based)
Variant reserved (NCS backward compatible)
{code}
which makes it unfortunately *INVALID*, due to the use of reserved characters
for the _Variant_
Assuming that the *POST *request is handled by:
{code:java}
org.apache.nifi.web.api.ProcessGroupResource#createProcessGroup()
{code}
the UUID is generated in line 2036
{code:java}
// set the processor id as appropriate
processGroup.setId(generateUuid());
{code}
which in turn is a call to the inherited method in the abstract class
ApplicationResource
{code:java}
protected
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361536#comment-17361536
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:14 AM:
--
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
A bit more background on when the issue arises.
I am using the nifi-api to create ProcessGroups. I use, the following REST POST
request:
{code:java}
curl --location --request POST
'http://localhost:9090/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
which returns
{code:json}
{
"revision": {
"version": 1,
"lastModifier": "anonymous"
},
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"uri":
"http://localhost:9090/nifi-api/process-groups/fa4829ef-0179-1000-5e44-7fc558cbb6d3;,
"position": {
"x": 0.0,
"y": 0.0
},
"permissions": {
"canRead": true,
"canWrite": true
},
"bulletins": [],
"component": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"parentGroupId": "58d527f9-0178-1000-21a9-cf75fa6606c0",
"position": {
"x": 0.0,
"y": 0.0
},
"name": "test",
"comments": "",
"variables": {},
"flowfileConcurrency": "UNBOUNDED",
"flowfileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"contents": {
"processGroups": [],
"remoteProcessGroups": [],
"processors": [],
"inputPorts": [],
"outputPorts": [],
"connections": [],
"labels": [],
"funnels": [],
"controllerServices": []
},
"inputPortCount": 0,
"outputPortCount": 0
},
"status": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"statsLastRefreshed": "08:55:04 UTC",
"aggregateSnapshot": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"flowFilesIn": 0,
"bytesIn": 0,
"input": "0 (0 bytes)",
"flowFilesQueued": 0,
"bytesQueued": 0,
"queued": "0 (0 bytes)",
"queuedCount": "0",
"queuedSize": "0 bytes",
"bytesRead": 0,
"read": "0 bytes",
"bytesWritten": 0,
"written": "0 bytes",
"flowFilesOut": 0,
"bytesOut": 0,
"output": "0 (0 bytes)",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "0 (0 bytes)",
"bytesReceived": 0,
"flowFilesReceived": 0,
"received": "0 (0 bytes)",
"bytesSent": 0,
"flowFilesSent": 0,
"sent": "0 (0 bytes)",
"activeThreadCount": 0,
"terminatedThreadCount": 0
}
},
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"inputPortCount": 0,
"outputPortCount": 0
}
{code}
Decoding using the [https://www.uuidtools.com/decode] we can see that the *id*
{{"id": "fa4829ef-0179-*1*000-*5*e44-7fc558cbb6d3"}}
has
{code:java}
Version 1 (time and node based)
Variant reserved (NCS backward compatible)
{code}
which makes it unfortunately *INVALID.*
Assuming that the *POST *request is handled by:
{code:java}
org.apache.nifi.web.api.ProcessGroupResource#createProcessGroup()
{code}
the UUID is generated in line 2036
{code:java}
// set the processor id as appropriate
processGroup.setId(generateUuid());
{code}
which in turn is a call to the inherited method in the abstract class
ApplicationResource
{code:java}
protected String generateUuid() {
final Optional seed =
[jira] [Comment Edited] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361536#comment-17361536
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 8:14 AM:
--
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
was (Author: aprakash):
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8669) Invalid UUIDs generated
[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361536#comment-17361536
]
Arun Prakash commented on NIFI-8669:
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
> Invalid UUIDs generated
> ---
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: Arun Prakash
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (MINIFICPP-994) Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it should be part of MiNiFi build.
[
https://issues.apache.org/jira/browse/MINIFICPP-994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Gyimesi reassigned MINIFICPP-994:
---
Assignee: Gabor Gyimesi (was: Nghia Le)
> Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it
> should be part of MiNiFi build.
>
>
> Key: MINIFICPP-994
> URL: https://issues.apache.org/jira/browse/MINIFICPP-994
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Minor
>
> Opencv is built when executing a cmake command (for eg. using bootstrap),
> which seems to be wrong as:
> * This build is single-threaded, takes a lot of time
> * Linker warnings are generated when building MiNiFI:
> {code}
> ld: warning: direct access in function '___cxx_global_var_init.35' from file
> '../thirdparty/opencv/opencv-external/src/opencv-external-build/lib/libopencv_flann.a(miniflann.cpp.o)'
> to global weak symbol 'guard variable for
> cvflann::anyimpl::SinglePolicy std::__1::char_traits, std::__1::allocator > >::policy' from file
> 'CMakeFiles/CaptureRTSPFrameTest.dir/CaptureRTSPFrameTest.cpp.o' means the
> weak symbol cannot be overridden at runtime. This was likely caused by
> different translation units being compiled with different visibility settings.
> {code}
> Static linking should be kept when integrating opencv build to the build
> process of MiNiFi.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8643) Nifi 1.13.2 Cluster Load balancer Does not Work Properly
[ https://issues.apache.org/jira/browse/NIFI-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361418#comment-17361418 ] Jens M Kofoed edited comment on NIFI-8643 at 6/11/21, 6:12 AM: --- Adding this line in the nifi.properties gets it to work (partly) nifi.cluster.load.balance.address=0.0.0.0 now the load.balance.port bind to 0.0.0.0 Now I can load balance between node 1 & 3. But I still gets a connection refuse to node2 Firewall is disabled and netstat shows that 6342 is bind to 0.0.0.0 There are a mismatch between nifi.properties and the java file handling the propertiesnifi.properties: nifi.cluster.load.balance.hostnifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java: nifi.cluster.load.balance.address kind regards Jens was (Author: jmkofoed): Adding this line in the nifi.properties gets it to work:nifi.cluster.load.balance.address=0.0.0.0 There are a mismatch between nifi.properties and the java file handling the propertiesnifi.properties: nifi.cluster.load.balance.hostnifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java: nifi.cluster.load.balance.address kind regards Jens > Nifi 1.13.2 Cluster Load balancer Does not Work Properly > - > > Key: NIFI-8643 > URL: https://issues.apache.org/jira/browse/NIFI-8643 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 > Environment: 3 node nifi cluster, installed in GCP Centos7 VM >Reporter: Kristian Mojeno >Priority: Major > Fix For: 1.12.1 > > > I have upgraded from nifi version 1.12.1 to 1.13.2. After the upgrade, the > cluster load balancer does not work properly and then flowfiles get stuck on > queue. Then the weird thing is that I can't see any error on logs. > I've tried to downgrade java from 11 down to version 8 but still same issue > exist. > My nifi cluster has three nodes deployed on a GCP VM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
