[jira] [Created] (NIFI-12624) ReportingTask: AWSCloudWatchReporterTask
Jorge Machado created NIFI-12624: Summary: ReportingTask: AWSCloudWatchReporterTask Key: NIFI-12624 URL: https://issues.apache.org/jira/browse/NIFI-12624 Project: Apache NiFi Issue Type: Wish Components: Core Framework Affects Versions: 1.24.0 Reporter: Jorge Machado Hey everyone, we already have a PrometheusReportingTask and an AzureLogAnalyticsReportingTask, it would be great if we could push all the metrics from PrometheusReportingTask into AWS Cloudwatch. What do you think ? Currently there is not straight up way of doing this. If yes where should we put it ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12351 Improve MiNiFi Service Scripts for Windows [nifi]
exceptionfactory commented on PR #8015: URL: https://github.com/apache/nifi/pull/8015#issuecomment-1894866499 @rliszli This seems close to completion, aside from the one merge conflict. @bejancsaba I will defer to you for further review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12402 add option to user can avoid an inactive indicator is crea… [nifi]
exceptionfactory commented on code in PR #8063:
URL: https://github.com/apache/nifi/pull/8063#discussion_r1454401906
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMonitorActivity.java:
##
@@ -349,6 +408,50 @@ public void testFirstRunNoMessages() throws
InterruptedException {
while(rerun);
}
+@Test
+public void testRunNoMessagesWithWaitForActivityTrue() throws
InterruptedException {
+// don't use the TestableProcessor, we want the real timestamp from
@OnScheduled
+final TestRunner runner = TestRunners.newTestRunner(new
MonitorActivity());
+runner.setProperty(MonitorActivity.CONTINUALLY_SEND_MESSAGES, "false");
+runner.setProperty(MonitorActivity.THRESHOLD, "5 secs");
+runner.setProperty(MonitorActivity.WAIT_FOR_ACTIVITY, "true");
+
+// shouldn't generate inactivity b/c run() will reset the
lastSuccessfulTransfer if @OnSchedule & onTrigger
+// does not get called more than MonitorActivity.THRESHOLD apart
+runner.run();
+runner.assertTransferCount(MonitorActivity.REL_SUCCESS, 0);
+runner.assertTransferCount(MonitorActivity.REL_INACTIVE, 0);
+runner.assertTransferCount(MonitorActivity.REL_ACTIVITY_RESTORED, 0);
+Thread.sleep(1L);
+runNext(runner);
+runner.assertTransferCount(MonitorActivity.REL_SUCCESS, 0);
+runner.assertTransferCount(MonitorActivity.REL_INACTIVE, 0);
+runner.assertTransferCount(MonitorActivity.REL_ACTIVITY_RESTORED, 0);
+runner.clearTransferState();
+}
+
+@Test
+public void testRunNoMessagesWithWaitForActivityFalse() throws
InterruptedException {
+// don't use the TestableProcessor, we want the real timestamp from
@OnScheduled
+final TestRunner runner = TestRunners.newTestRunner(new
MonitorActivity());
+runner.setProperty(MonitorActivity.CONTINUALLY_SEND_MESSAGES, "false");
+runner.setProperty(MonitorActivity.THRESHOLD, "5 secs");
+runner.setProperty(MonitorActivity.WAIT_FOR_ACTIVITY, "false");
+
+// shouldn't generate inactivity b/c run() will reset the
lastSuccessfulTransfer if @OnSchedule & onTrigger
+// does not get called more than MonitorActivity.THRESHOLD apart
+runner.run();
+runner.assertTransferCount(MonitorActivity.REL_SUCCESS, 0);
+runner.assertTransferCount(MonitorActivity.REL_INACTIVE, 0);
+runner.assertTransferCount(MonitorActivity.REL_ACTIVITY_RESTORED, 0);
+Thread.sleep(1L);
+runNext(runner);
+runner.assertTransferCount(MonitorActivity.REL_SUCCESS, 0);
+runner.assertTransferCount(MonitorActivity.REL_INACTIVE, 1);
+runner.assertTransferCount(MonitorActivity.REL_ACTIVITY_RESTORED, 0);
+runner.clearTransferState();
+}
+
Review Comment:
The sleep of 10 seconds is too long for efficient unit testing. The other
unit test method provides some basic coverage, so recommend removing these two
test methods.
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMonitorActivity.java:
##
@@ -100,6 +100,65 @@ public void testFirstMessage() {
restoredFlowFile.assertAttributeNotExists("key1");
}
+@Test
+public void testFirstMessageWithWaitForActivityTrue() {
+final TestableProcessor processor = new TestableProcessor(1000);
+final TestRunner runner = TestRunners.newTestRunner(processor);
+runner.setProperty(MonitorActivity.CONTINUALLY_SEND_MESSAGES, "false");
+runner.setProperty(MonitorActivity.THRESHOLD, "100 millis");
+runner.setProperty(MonitorActivity.WAIT_FOR_ACTIVITY, "true");
+
+runner.enqueue(new byte[0]);
+runner.run();
+runner.assertAllFlowFilesTransferred(MonitorActivity.REL_SUCCESS, 1);
+runner.clearTransferState();
+
+processor.resetLastSuccessfulTransfer();
+
+runNext(runner);
+runner.assertAllFlowFilesTransferred(MonitorActivity.REL_INACTIVE, 1);
+runner.clearTransferState();
+
+Map attributes = new HashMap<>();
+attributes.put("key", "value");
+attributes.put("key1", "value1");
+
+runner.enqueue(new byte[0], attributes);
+runNext(runner);
+
+runner.assertTransferCount(MonitorActivity.REL_SUCCESS, 1);
+runner.assertTransferCount(MonitorActivity.REL_ACTIVITY_RESTORED, 1);
+
+MockFlowFile restoredFlowFile =
runner.getFlowFilesForRelationship(MonitorActivity.REL_ACTIVITY_RESTORED).get(0);
+String flowFileContent = new String(restoredFlowFile.toByteArray());
+assertTrue(Pattern.matches("Activity restored at time: (.*) after
being inactive for 0 minutes", flowFileContent));
Review Comment:
It is best to avoid checking particular messages, as these are not
[jira] [Commented] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807512#comment-17807512
]
ASF subversion and git services commented on NIFI-8278:
---
Commit e1f920cd2bf1550928ff13c044155a51ac88ffe2 in nifi's branch
refs/heads/main from Peter Turcsanyi
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e1f920cd2b ]
NIFI-8278 Fixed Proxy Service property in Azure Storage processors
This closes #8259
Signed-off-by: David Handermann
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 2.0.0-M2
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-8278 Fixed Proxy Configuration Service property in Azure storage… [nifi]
exceptionfactory closed pull request #8259: NIFI-8278 Fixed Proxy Configuration Service property in Azure storage… URL: https://github.com/apache/nifi/pull/8259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12623) Allow ListenSlack to receive App Mention events and include user details
[ https://issues.apache.org/jira/browse/NIFI-12623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807506#comment-17807506 ] ASF subversion and git services commented on NIFI-12623: Commit aa837853d1fab5719dc32589a0a3e62460f16019 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=aa837853d1 ] NIFI-12623: Expose ability to fetch User Details in ListenSlack and receive App Mention events This closes #8258 Signed-off-by: David Handermann > Allow ListenSlack to receive App Mention events and include user details > > > Key: NIFI-12623 > URL: https://issues.apache.org/jira/browse/NIFI-12623 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0-M2 > > Time Spent: 20m > Remaining Estimate: 0h > > When using the ListenSlack processor, I often want only the events that > mention my bot by name. However, as it is, the processor requires that I > receive all events in the channels that have my bot, and then filter them > out. We should instead allow users to receive only App Mention events. > Additionally, it would be beneficial to retrieve user details such as > username instead of just the User ID. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12623) Allow ListenSlack to receive App Mention events and include user details
[ https://issues.apache.org/jira/browse/NIFI-12623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12623. - Resolution: Fixed > Allow ListenSlack to receive App Mention events and include user details > > > Key: NIFI-12623 > URL: https://issues.apache.org/jira/browse/NIFI-12623 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0-M2 > > Time Spent: 20m > Remaining Estimate: 0h > > When using the ListenSlack processor, I often want only the events that > mention my bot by name. However, as it is, the processor requires that I > receive all events in the channels that have my bot, and then filter them > out. We should instead allow users to receive only App Mention events. > Additionally, it would be beneficial to retrieve user details such as > username instead of just the User ID. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12623: Expose ability to fetch User Details in ListenSlack and r… [nifi]
exceptionfactory closed pull request #8258: NIFI-12623: Expose ability to fetch User Details in ListenSlack and r… URL: https://github.com/apache/nifi/pull/8258 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11294 ConsumeAzureEventHub supports storing checkpoints in compo… [nifi]
exceptionfactory commented on code in PR #8013:
URL: https://github.com/apache/nifi/pull/8013#discussion_r1454350276
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/checkpoint/CheckpointConstants.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */package org.apache.nifi.processors.azure.eventhub.checkpoint;
+
+public final class CheckpointConstants {
Review Comment:
Rather than having a generalized `Constants` class, it is better to use one
or more enums that provide a clear purpose. In this case, I recommend a
`CheckpointStateKey` enum for `CLIENT_ID` and `CLUSTERED`, and a
`StoreKeyPrefix` enum for `OWNERSHIP` and `CHECKPOINT`. In the case of
`StoreKeyPrefix`, the enum could have a method named `getPrefix()` that would
return the lowercase value.
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/checkpoint/CheckpointConstants.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */package org.apache.nifi.processors.azure.eventhub.checkpoint;
Review Comment:
The `package` line should start on a new line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-11958) Add PutAzureDataExplorer and Service for Microsoft Azure Data Explorer Integration
[ https://issues.apache.org/jira/browse/NIFI-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11958. - Fix Version/s: 2.0.0-M2 Assignee: Tanmaya Panda Resolution: Fixed > Add PutAzureDataExplorer and Service for Microsoft Azure Data Explorer > Integration > -- > > Key: NIFI-11958 > URL: https://issues.apache.org/jira/browse/NIFI-11958 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Tanmaya Panda >Assignee: Tanmaya Panda >Priority: Minor > Fix For: 2.0.0-M2 > > Time Spent: 4h > Remaining Estimate: 0h > > Add new processor which connects and sends data to Microsoft Azure Data > Explorer ingestion pipeline. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11958) Add PutAzureDataExplorer and Service for Microsoft Azure Data Explorer Integration
[ https://issues.apache.org/jira/browse/NIFI-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11958: Summary: Add PutAzureDataExplorer and Service for Microsoft Azure Data Explorer Integration (was: Add Sink Processor and Service for Microsoft Azure Data Explorer Integration) > Add PutAzureDataExplorer and Service for Microsoft Azure Data Explorer > Integration > -- > > Key: NIFI-11958 > URL: https://issues.apache.org/jira/browse/NIFI-11958 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Tanmaya Panda >Priority: Minor > Time Spent: 4h > Remaining Estimate: 0h > > Add new processor which connects and sends data to Microsoft Azure Data > Explorer ingestion pipeline. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11958) Add Sink Processor and Service for Microsoft Azure Data Explorer Integration
[ https://issues.apache.org/jira/browse/NIFI-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11958: Summary: Add Sink Processor and Service for Microsoft Azure Data Explorer Integration (was: Sink Processor and Service for Microsoft Azure Data Explorer Integration) > Add Sink Processor and Service for Microsoft Azure Data Explorer Integration > > > Key: NIFI-11958 > URL: https://issues.apache.org/jira/browse/NIFI-11958 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Tanmaya Panda >Priority: Minor > Time Spent: 4h > Remaining Estimate: 0h > > Add new processor which connects and sends data to Microsoft Azure Data > Explorer ingestion pipeline. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11958) Sink Processor and Service for Microsoft Azure Data Explorer Integration
[ https://issues.apache.org/jira/browse/NIFI-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807501#comment-17807501 ] ASF subversion and git services commented on NIFI-11958: Commit a613c524373f475fdd7576b88557286622bd2b4f in nifi's branch refs/heads/main from Tanmaya Panda [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a613c52437 ] NIFI-11958 Added PutAzureDataExplorer and StandardKustoQueryService This closes #7624 Signed-off-by: David Handermann > Sink Processor and Service for Microsoft Azure Data Explorer Integration > > > Key: NIFI-11958 > URL: https://issues.apache.org/jira/browse/NIFI-11958 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Tanmaya Panda >Priority: Minor > Time Spent: 4h > Remaining Estimate: 0h > > Add new processor which connects and sends data to Microsoft Azure Data > Explorer ingestion pipeline. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11958 Azure Data Explorer as sink [nifi]
exceptionfactory closed pull request #7624: NIFI-11958 Azure Data Explorer as sink URL: https://github.com/apache/nifi/pull/7624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
exceptionfactory commented on code in PR #8240:
URL: https://github.com/apache/nifi/pull/8240#discussion_r1454335733
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/src/main/java/org/apache/nifi/kubernetes/state/provider/KubernetesConfigMapStateProvider.java:
##
@@ -99,6 +118,12 @@ public void initialize(final
StateProviderInitializationContext context) {
this.logger = context.getLogger();
this.kubernetesClient = getKubernetesClient();
this.namespace = new ServiceAccountNamespaceProvider().getNamespace();
+
+String configMapNamePrefix =
context.getProperty(CONFIG_MAP_NAME_PREFIX).isSet() ?
context.getProperty(CONFIG_MAP_NAME_PREFIX).getValue() : null;
+CONFIG_MAP_NAME_FORMAT = configMapNamePrefix != null
+? String.format("%s-%s-%%s", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("%s-%%s", CONFIG_MAP_CORE_NAME);
+CONFIG_MAP_NAME_PATTERN = Pattern.compile(configMapNamePrefix != null
+? String.format("^%s-%s-(.+)$", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("^%s-(.+)$", CONFIG_MAP_CORE_NAME));
Review Comment:
Thanks for making the adjustments @juldrixx, the latest version looks much
cleaner!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
exceptionfactory commented on code in PR #8240: URL: https://github.com/apache/nifi/pull/8240#discussion_r1454334009 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/pom.xml: ## @@ -48,5 +48,10 @@ kubernetes-server-mock test + +org.apache.nifi +nifi-expression-language Review Comment: Thanks for the clarification, that makes sense given that `nifi-expression-language` is also a dependency of the more general `nifi-mock` test library. This looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11294 ConsumeAzureEventHub supports storing checkpoints in compo… [nifi]
turcsanyip commented on PR #8013: URL: https://github.com/apache/nifi/pull/8013#issuecomment-1894716050 @exceptionfactory Thanks for the clarification on temporarily disconnected nodes due cluster communication problems vs manual disconnection! Good to know that a temporarily disconnected node behaves differently. Regarding the manual disconnection, I found a relatively easy way to check it: - add `isClustered` entry in the `LOCAL` scope, set it only when it is unset (first start-up or after manual state clean-up) - check its value in the `CLUSTER` scope (from where the checkpoints are retrieved) - if the entry is not found, it is fine as we are really accessing the cluster scope, so the node is currently connected to the cluster - if the entry is found and it is `true`, then it is an originally clustered node but is currently accessing the local state via the cluster scope => the node is disconnected and it must not write the state - if the entry is found and it is `false`, then it is a standalone node => OK I added this change along with 2 other commits improving tests and error handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
juldrixx commented on code in PR #8240:
URL: https://github.com/apache/nifi/pull/8240#discussion_r1454215788
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/src/main/java/org/apache/nifi/kubernetes/state/provider/KubernetesConfigMapStateProvider.java:
##
@@ -99,6 +118,12 @@ public void initialize(final
StateProviderInitializationContext context) {
this.logger = context.getLogger();
this.kubernetesClient = getKubernetesClient();
this.namespace = new ServiceAccountNamespaceProvider().getNamespace();
+
+String configMapNamePrefix =
context.getProperty(CONFIG_MAP_NAME_PREFIX).isSet() ?
context.getProperty(CONFIG_MAP_NAME_PREFIX).getValue() : null;
+CONFIG_MAP_NAME_FORMAT = configMapNamePrefix != null
+? String.format("%s-%s-%%s", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("%s-%%s", CONFIG_MAP_CORE_NAME);
+CONFIG_MAP_NAME_PATTERN = Pattern.compile(configMapNamePrefix != null
+? String.format("^%s-%s-(.+)$", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("^%s-(.+)$", CONFIG_MAP_CORE_NAME));
Review Comment:
I made the change. I let you confirm that it was what you wanted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
juldrixx commented on code in PR #8240: URL: https://github.com/apache/nifi/pull/8240#discussion_r1454206171 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/pom.xml: ## @@ -48,5 +48,10 @@ kubernetes-server-mock test + +org.apache.nifi +nifi-expression-language Review Comment: It's needed for ```java when(context.getProperty(KubernetesConfigMapStateProvider.CONFIG_MAP_NAME_PREFIX)) .thenReturn(new StandardPropertyValue(null, null, ParameterLookup.EMPTY)); ``` and ```java when(context.getProperty(KubernetesConfigMapStateProvider.CONFIG_MAP_NAME_PREFIX)) .thenReturn(new StandardPropertyValue(CONFIG_MAP_NAME_PREFIX_VALUE, null, ParameterLookup.EMPTY)); ``` For `StandardPropertyValue` and `ParameterLookup`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
exceptionfactory commented on code in PR #8240:
URL: https://github.com/apache/nifi/pull/8240#discussion_r1454177876
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/src/main/java/org/apache/nifi/kubernetes/state/provider/KubernetesConfigMapStateProvider.java:
##
@@ -99,6 +118,12 @@ public void initialize(final
StateProviderInitializationContext context) {
this.logger = context.getLogger();
this.kubernetesClient = getKubernetesClient();
this.namespace = new ServiceAccountNamespaceProvider().getNamespace();
+
+String configMapNamePrefix =
context.getProperty(CONFIG_MAP_NAME_PREFIX).isSet() ?
context.getProperty(CONFIG_MAP_NAME_PREFIX).getValue() : null;
+CONFIG_MAP_NAME_FORMAT = configMapNamePrefix != null
+? String.format("%s-%s-%%s", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("%s-%%s", CONFIG_MAP_CORE_NAME);
+CONFIG_MAP_NAME_PATTERN = Pattern.compile(configMapNamePrefix != null
+? String.format("^%s-%s-(.+)$", configMapNamePrefix,
CONFIG_MAP_CORE_NAME) : String.format("^%s-(.+)$", CONFIG_MAP_CORE_NAME));
Review Comment:
There is still a bit of repetition here that would be helpful to streamline,
given the importance of proper formatting. The ternary expression is also
difficult to read. I recommend the `PATTERN_FORMAT` as a static value with
`^%snifi-component-(.+)$` and then supplying either an empty string, or the
configured prefix to build the final Pattern. Declaring the intermediate values
could also make this more readable.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/pom.xml:
##
@@ -269,6 +269,9 @@
100
10
./logs
+
+
Review Comment:
The comment can be removed as it does not provide additional information.
```suggestion
```
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/src/main/java/org/apache/nifi/kubernetes/state/provider/KubernetesConfigMapStateProvider.java:
##
@@ -70,6 +78,10 @@ public class KubernetesConfigMapStateProvider extends
AbstractConfigurableCompon
private final AtomicBoolean enabled = new AtomicBoolean();
+private String CONFIG_MAP_NAME_FORMAT;
+
+private Pattern CONFIG_MAP_NAME_PATTERN;
Review Comment:
These should be changed reflect that they are member variables.
```suggestion
private String configMapNameFormat;
private Pattern configMapNamePattern;
```
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-kubernetes-bundle/nifi-framework-kubernetes-state-provider/pom.xml:
##
@@ -48,5 +48,10 @@
kubernetes-server-mock
test
+
+org.apache.nifi
+nifi-expression-language
Review Comment:
Is this dependency required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12554 Placed all Jolt related code under a new nifi-jolt-bundle and refactored to reduce duplicate code. [nifi]
dan-s1 commented on PR #8249: URL: https://github.com/apache/nifi/pull/8249#issuecomment-1894633928 @exceptionfactory I am now able to see the stacktraces from the builds. Apparently there was also an issue with the `nifi-standard-utils` which no longer exists. So I removed both `nifi-standard-utils` and `nifi-jolt-record-processors` and replaced them with `nifi-jolt-processors` and `nifi-jolt-utils`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12596) PutIceberg is missing case-insensitive Record type handling in List and Map types
[ https://issues.apache.org/jira/browse/NIFI-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807476#comment-17807476 ] ASF subversion and git services commented on NIFI-12596: Commit c92034cda6f33f83189ae3cf0275b770770a9e3d in nifi's branch refs/heads/main from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c92034cda6 ] NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types Signed-off-by: Matt Burgess This closes #8234 > PutIceberg is missing case-insensitive Record type handling in List and Map > types > - > > Key: NIFI-12596 > URL: https://issues.apache.org/jira/browse/NIFI-12596 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > With NIFI-11263 case-insensitive and order independent field handling was > added to Record types but it is missing in case of List and Map types > containing Records. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12596) PutIceberg is missing case-insensitive Record type handling in List and Map types
[ https://issues.apache.org/jira/browse/NIFI-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12596: Fix Version/s: 1.25.0 2.0.0-M2 Resolution: Fixed Status: Resolved (was: Patch Available) > PutIceberg is missing case-insensitive Record type handling in List and Map > types > - > > Key: NIFI-12596 > URL: https://issues.apache.org/jira/browse/NIFI-12596 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > With NIFI-11263 case-insensitive and order independent field handling was > added to Record types but it is missing in case of List and Map types > containing Records. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-8278 Fixed Proxy Configuration Service property in Azure storage… [nifi]
turcsanyip opened a new pull request, #8259: URL: https://github.com/apache/nifi/pull/8259 … processors # Summary [NIFI-8278](https://issues.apache.org/jira/browse/NIFI-8278) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types [nifi]
mattyb149 closed pull request #8234: NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types URL: https://github.com/apache/nifi/pull/8234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types [nifi]
mattyb149 commented on PR #8234: URL: https://github.com/apache/nifi/pull/8234#issuecomment-1894626616 +1 LGTM, tested with and without the PR and verified the correct behavior. Thanks for the fix! Merging to support/nifi-1.x and main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12596) PutIceberg is missing case-insensitive Record type handling in List and Map types
[ https://issues.apache.org/jira/browse/NIFI-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807473#comment-17807473 ] ASF subversion and git services commented on NIFI-12596: Commit 99ffcdb3bbcdeb284679b8f381b1fed08d98a971 in nifi's branch refs/heads/support/nifi-1.x from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=99ffcdb3bb ] NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types Signed-off-by: Matt Burgess > PutIceberg is missing case-insensitive Record type handling in List and Map > types > - > > Key: NIFI-12596 > URL: https://issues.apache.org/jira/browse/NIFI-12596 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > With NIFI-11263 case-insensitive and order independent field handling was > added to Record types but it is missing in case of List and Map types > containing Records. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12623: Expose ability to fetch User Details in ListenSlack and r… [nifi]
markap14 opened a new pull request, #8258: URL: https://github.com/apache/nifi/pull/8258 …eceive App Mention events # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-11669) Allow expired entries in distributed cache to be pruned
[ https://issues.apache.org/jira/browse/NIFI-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bean resolved NIFI-11669. -- Resolution: Won't Fix > Allow expired entries in distributed cache to be pruned > --- > > Key: NIFI-11669 > URL: https://issues.apache.org/jira/browse/NIFI-11669 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.21.0 >Reporter: Mark Bean >Priority: Major > > DetectDuplicate processor adds entries to a DistributedMapCacheClient > service. The processor also has an Age Off Duration property. The cached > value is removed from the cache only if a subsequent duplicate is detected > and the (optional) Age Off Duration has expired. The result is that entries > which are beyond their age off duration remain in the cache needlessly and > continue to consume memory. > There should be a proactive pruning process to eliminate the expired entries. > Two thoughts come to mind: using a DistributedMapCache implementation which > expires entries. This is the most flexible as other users of the cache would > benefit as well. Alternatively, the DetectDuplicate processor could > periodically prune the cache. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12623) Allow ListenSlack to receive App Mention events and include user details
Mark Payne created NIFI-12623: - Summary: Allow ListenSlack to receive App Mention events and include user details Key: NIFI-12623 URL: https://issues.apache.org/jira/browse/NIFI-12623 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Mark Payne Assignee: Mark Payne Fix For: 2.0.0-M2 When using the ListenSlack processor, I often want only the events that mention my bot by name. However, as it is, the processor requires that I receive all events in the channels that have my bot, and then filter them out. We should instead allow users to receive only App Mention events. Additionally, it would be beneficial to retrieve user details such as username instead of just the User ID. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807468#comment-17807468
]
Peter Turcsanyi commented on NIFI-8278:
---
Thanks for the heads-up [~bbende]! Yes, it is due to
PROXY_CONFIGURATION_SERVICE because it is used by both the controller services
and the processors but I forgot about the latter and added dependsOn based on
the usage in the services. 'credentials-type' is a new property in the ADLS
service and it was never present on the processors. Just the
PROXY_CONFIGURATION_SERVICE property definition was shared. Will fix it in new
PR.
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 2.0.0-M2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-9458 Replace SimpleDateFormat with DateTimeFormatter [nifi]
bbende merged PR #8248: URL: https://github.com/apache/nifi/pull/8248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9458) Refactor NiFi to use Java 8 Time classes
[ https://issues.apache.org/jira/browse/NIFI-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807457#comment-17807457 ] ASF subversion and git services commented on NIFI-9458: --- Commit 250fe90b348fac515ea597c1985ca432ac7c3ac3 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=250fe90b34 ] NIFI-9458 Replaced SimpleDateFormat with DateTimeFormatter (#8248) - Added FieldConverterRegistry and Standard implementation - Moved DateTypeUtils methods to FieldConverter implementations > Refactor NiFi to use Java 8 Time classes > > > Key: NIFI-9458 > URL: https://issues.apache.org/jira/browse/NIFI-9458 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Extensions >Reporter: Pierre Villard >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M2 > > Time Spent: 40m > Remaining Estimate: 0h > > In order to support microseconds or even nanoseconds in NiFi, we should > consider refactoring NiFi code to leverage Java 8 Time classes. This is going > to be a significant amount of work so creating this epic to breakdown the > work into smaller pieces. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-9458) Refactor NiFi to use Java 8 Time classes
[ https://issues.apache.org/jira/browse/NIFI-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-9458: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Refactor NiFi to use Java 8 Time classes > > > Key: NIFI-9458 > URL: https://issues.apache.org/jira/browse/NIFI-9458 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Extensions >Reporter: Pierre Villard >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M2 > > Time Spent: 40m > Remaining Estimate: 0h > > In order to support microseconds or even nanoseconds in NiFi, we should > consider refactoring NiFi code to leverage Java 8 Time classes. This is going > to be a significant amount of work so creating this epic to breakdown the > work into smaller pieces. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12554 Placed all Jolt related code under a new nifi-jolt-bundle and refactored to reduce duplicate code. [nifi]
exceptionfactory commented on PR #8249: URL: https://github.com/apache/nifi/pull/8249#issuecomment-1894557055 > @exceptionfactory Do I need to add the new modules I created to `nifi-code-coverage` `pom.xml`? Which ones do I add `nifi-jolt-processors` and `nifi-jolt-utils` ? Yes, those two should be added and the old one should be removed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12554 Placed all Jolt related code under a new nifi-jolt-bundle and refactored to reduce duplicate code. [nifi]
dan-s1 commented on PR #8249: URL: https://github.com/apache/nifi/pull/8249#issuecomment-1894548409 @exceptionfactory Do I need to add the new modules I created to `nifi-code-coverage` `pom.xml`? Which ones do I add `nifi-jolt-processors` and `nifi-jolt-utils` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11958 Azure Data Explorer as sink [nifi]
exceptionfactory commented on PR #7624: URL: https://github.com/apache/nifi/pull/7624#issuecomment-1894538604 Rebased to resolve merge conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers (NIFI-12622) * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI ** Property History ** Unable to re-add any removed Property * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG * Configure RPG * RPG Remote Ports * RPG Go To * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Better theme support * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Routing error handling * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Prompt user to save Parameter Context when Edit form is dirty * Start/Stop processors, process groups, ... (NIFI-12568) * Dialog vertical resizing on smaller screens do not allow users to access all fields (NIFI-12603) [1] [https://github.com/apache/nifi/pull/8053] was: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services
[jira] [Created] (NIFI-12622) Parameter Providers
Rob Fellows created NIFI-12622: -- Summary: Parameter Providers Key: NIFI-12622 URL: https://issues.apache.org/jira/browse/NIFI-12622 Project: Apache NiFi Issue Type: Sub-task Reporter: Rob Fellows Assignee: Rob Fellows -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12615) BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page.
[ https://issues.apache.org/jira/browse/NIFI-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807435#comment-17807435 ] ASF subversion and git services commented on NIFI-12615: Commit b47ca20f56d45a1f52f2cdf24eb7d218eb7aeaaf in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b47ca20f56 ] [NIFI-12615] - fix ExpressionChanged error on Counters page. (#8252) * [NIFI-12615] - fix ExpressionChanged error on Counters page. * * refactor sorting for extension-creation.component * refactor sorting for controller-service-table.component * refactor sorting for reporting-task--table.component * refactor sorting for parameter-context-table.component This closes #8252 > BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page. > -- > > Key: NIFI-12615 > URL: https://issues.apache.org/jira/browse/NIFI-12615 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-01-16 at 8.47.12 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > We've seen this same error regarding tables where we are trying to support > initial sorting for a table and then leverage the even/odd variables to class > the rows. The solution is to implement our own sorting and not use the > MatSort. > Also, as part of this we should update other cases where we rely on > MatSort/sortingDataAccessor do do our sorting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12615) BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page.
[ https://issues.apache.org/jira/browse/NIFI-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12615: --- Fix Version/s: 2.0.0-M2 Resolution: Fixed Status: Resolved (was: Patch Available) > BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page. > -- > > Key: NIFI-12615 > URL: https://issues.apache.org/jira/browse/NIFI-12615 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0-M2 > > Attachments: Screenshot 2024-01-16 at 8.47.12 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > We've seen this same error regarding tables where we are trying to support > initial sorting for a table and then leverage the even/odd variables to class > the rows. The solution is to implement our own sorting and not use the > MatSort. > Also, as part of this we should update other cases where we rely on > MatSort/sortingDataAccessor do do our sorting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12615] - fix ExpressionChanged error on Counters page. [nifi]
mcgilman merged PR #8252: URL: https://github.com/apache/nifi/pull/8252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12621) Upgrade AWS SDK to 2.23.3 and 1.12.637
[ https://issues.apache.org/jira/browse/NIFI-12621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12621: Status: Patch Available (was: Open) > Upgrade AWS SDK to 2.23.3 and 1.12.637 > -- > > Key: NIFI-12621 > URL: https://issues.apache.org/jira/browse/NIFI-12621 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 10m > Remaining Estimate: 0h > > The AWS SDK libraries should be upgraded to the latest available versions for > SDK 1 and 2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12621 Upgraded AWS SDK from 2.20.148 to 2.23.3 [nifi]
exceptionfactory opened a new pull request, #8257: URL: https://github.com/apache/nifi/pull/8257 # Summary [NIFI-12621](https://issues.apache.org/jira/browse/NIFI-12621) Upgrades AWS SDK versions from 2.20.148 to 2.23.3 and from 1.12.573 to 1.12.637. Additional updates include a dependency check suppression to avoid identifying the newer Amazon `ion-java` library with the older version, which has an associated vulnerability finding. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12621) Upgrade AWS SDK to 2.23.3 and 1.12.637
David Handermann created NIFI-12621: --- Summary: Upgrade AWS SDK to 2.23.3 and 1.12.637 Key: NIFI-12621 URL: https://issues.apache.org/jira/browse/NIFI-12621 Project: Apache NiFi Issue Type: Improvement Reporter: David Handermann Assignee: David Handermann The AWS SDK libraries should be upgraded to the latest available versions for SDK 1 and 2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807420#comment-17807420
]
Bryan Bende commented on NIFI-8278:
---
After this commit I noticed the following in the logs during start up for
most/all of the Azure processors...
{code:java}
WARN [main] o.apache.nifi.documentation.DocGenerator Documentation generation
failed: Component Class [class
org.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12]
java.io.IOException: Unable to create XMLOutputStream
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:103)
at
org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
at
org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
at org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:745)
at org.apache.nifi.NiFi.(NiFi.java:172)
at org.apache.nifi.NiFi.(NiFi.java:83)
at org.apache.nifi.NiFi.main(NiFi.java:332)
Caused by: javax.xml.stream.XMLStreamException: No property was found matching
the name 'credentials-type'
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:744)
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:167)
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:99)
... 7 common frames omitted {code}
I'm not totally sure, but I think the problem might be that the
PROXY_CONFIGURATION_SERVICE is defined as a property in the processors with a
dependsOn the credentials-type which is no longer a property of the processor.
The PROXY_CONFIGURATION_SERVICE is also a property inside the credentials
controller service, so maybe it doesn't belong on the processors anymore?
[~turcsanyip] [~exceptionfactory] ?
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 2.0.0-M2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12615) BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page.
[ https://issues.apache.org/jira/browse/NIFI-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12615: --- Status: Patch Available (was: In Progress) > BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page. > -- > > Key: NIFI-12615 > URL: https://issues.apache.org/jira/browse/NIFI-12615 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-01-16 at 8.47.12 AM.png > > Time Spent: 20m > Remaining Estimate: 0h > > We've seen this same error regarding tables where we are trying to support > initial sorting for a table and then leverage the even/odd variables to class > the rows. The solution is to implement our own sorting and not use the > MatSort. > Also, as part of this we should update other cases where we rely on > MatSort/sortingDataAccessor do do our sorting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12618) Upgrade Azure SDK BOM to 1.2.19
[ https://issues.apache.org/jira/browse/NIFI-12618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807416#comment-17807416 ] ASF subversion and git services commented on NIFI-12618: Commit 26c21bbdcc22b926db404a65cc4e1685dff7689d in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=26c21bbdcc ] NIFI-12618 Upgraded Azure SDK BOM from 1.2.18 to 1.2.19 - Suppressed CVE-2023-36052 not applicable to Java Azure libraries Signed-off-by: Pierre Villard This closes #8254. > Upgrade Azure SDK BOM to 1.2.19 > --- > > Key: NIFI-12618 > URL: https://issues.apache.org/jira/browse/NIFI-12618 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The Azure SDK BOM should be upgraded from 1.2.18 to 1.2.19 to incorporate > incremental updates to Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12618) Upgrade Azure SDK BOM to 1.2.19
[ https://issues.apache.org/jira/browse/NIFI-12618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12618: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Azure SDK BOM to 1.2.19 > --- > > Key: NIFI-12618 > URL: https://issues.apache.org/jira/browse/NIFI-12618 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The Azure SDK BOM should be upgraded from 1.2.18 to 1.2.19 to incorporate > incremental updates to Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12620) Upgrade JLine to 3.25.0
[ https://issues.apache.org/jira/browse/NIFI-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12620: Status: Patch Available (was: Open) > Upgrade JLine to 3.25.0 > --- > > Key: NIFI-12620 > URL: https://issues.apache.org/jira/browse/NIFI-12620 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 10m > Remaining Estimate: 0h > > JLine > [3.25.0|https://github.com/jline/jline3/releases/tag/jline-parent-3.25.0] > includes several bug fixes and a resolution for CVE-2023-50572, which impacts > GroovyEngine usage. NiFi Toolkit dependencies on JLine do not use the > GroovyEngine for execution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12620 Upgrade JLine from 3.24.1 to 3.25.0 [nifi]
exceptionfactory opened a new pull request, #8256: URL: https://github.com/apache/nifi/pull/8256 # Summary [NIFI-12620](https://issues.apache.org/jira/browse/NIFI-12620) Upgrades JLine dependencies in the Toolkit CLI from 3.24.1 to 3.25.0, resolving CVE-2023-50572, which impacts GroovyEngine usage. NiFi Toolkit CLI use of JLine does not include references to the GroovyEngine class. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12620) Upgrade JLine to 3.25.0
David Handermann created NIFI-12620: --- Summary: Upgrade JLine to 3.25.0 Key: NIFI-12620 URL: https://issues.apache.org/jira/browse/NIFI-12620 Project: Apache NiFi Issue Type: Improvement Reporter: David Handermann Assignee: David Handermann Fix For: 1.25.0, 2.0.0-M2 JLine [3.25.0|https://github.com/jline/jline3/releases/tag/jline-parent-3.25.0] includes several bug fixes and a resolution for CVE-2023-50572, which impacts GroovyEngine usage. NiFi Toolkit dependencies on JLine do not use the GroovyEngine for execution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12618) Upgrade Azure SDK BOM to 1.2.19
[ https://issues.apache.org/jira/browse/NIFI-12618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807414#comment-17807414 ] ASF subversion and git services commented on NIFI-12618: Commit c6f5f534cb40c1d9d1ac1db458eb4f4567e40eaa in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c6f5f534cb ] NIFI-12618 Upgraded Azure SDK BOM from 1.2.18 to 1.2.19 - Suppressed CVE-2023-36052 not applicable to Java Azure libraries Signed-off-by: Pierre Villard This closes #8254. > Upgrade Azure SDK BOM to 1.2.19 > --- > > Key: NIFI-12618 > URL: https://issues.apache.org/jira/browse/NIFI-12618 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 20m > Remaining Estimate: 0h > > The Azure SDK BOM should be upgraded from 1.2.18 to 1.2.19 to incorporate > incremental updates to Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12618 Upgrade Azure SDK BOM from 1.2.18 to 1.2.19 [nifi]
asfgit closed pull request #8254: NIFI-12618 Upgrade Azure SDK BOM from 1.2.18 to 1.2.19 URL: https://github.com/apache/nifi/pull/8254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12619) Unable to instantiate ParseDocument processor
[
https://issues.apache.org/jira/browse/NIFI-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12619:
--
Description:
Trying to instantiate the Python processor ParseDocument, I get this error:
{code:java}
2024-01-16 18:00:00,573 INFO org.apache.nifi.py4j.ExtensionManager Importing
dependencies ['langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'] for
ParseDocument to
/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT
using command
['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
'langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx']
2024-01-16 18:00:15,752 ERROR py4j.java_gateway There was an exception while
executing the Python Proxy on the Python Side.
Traceback (most recent call last):
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/py4j/java_gateway.py",
line 2466, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
^^^
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./python/framework/Controller.py",
line 72, in downloadDependencies
self.extensionManager.import_external_dependencies(processor_details,
work_dir)
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/ExtensionManager.py",
line 511, in import_external_dependencies
raise RuntimeError(f"Failed to import requirements for {class_name}:
process exited with status code {result}")
RuntimeError: Failed to import requirements for ParseDocument: process exited
with status code
CompletedProcess(args=['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
'langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'], returncode=1)
{code}
If trying to run the pip command manually, I get
{code:java}
no matches found: pdfminer.six[image] {code}
Changing the required dependency to just *pdfminer.six* fixes the issue and I
can instantiate the processor.
However when trying to use it against a PDF file, I get:
{code:java}
ModuleNotFoundError: No module named 'pikepdf'
ModuleNotFoundError: No module named 'pypdf'{code}
After adding the above dependencies, I get:
{code:java}
pdf2image.exceptions.PDFInfoNotInstalledError: Unable to get page count. Is
poppler installed and in PATH? {code}
Based on
[https://pdf2image.readthedocs.io/en/latest/installation.html]
It sounds like poppler would need to be installed separately. I did it with
brew for my local instance. -Probably worth adding this in the docs if doable.-
This is specified in the description of the processor.
At this point I was able to use the processor to parse a PDF file.
was:
Trying to instantiate the Python processor ParseDocument, I get this error:
{code:java}
2024-01-16 18:00:00,573 INFO org.apache.nifi.py4j.ExtensionManager Importing
dependencies ['langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'] for
ParseDocument to
/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT
using command
['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
'langchain', 'unstructured', 'unstructured-inference',
[PR] NIFI-12619 - Fix python dependencies in ParseDocument [nifi]
pvillard31 opened a new pull request, #8255: URL: https://github.com/apache/nifi/pull/8255 # Summary [NIFI-12619](https://issues.apache.org/jira/browse/NIFI-12619) - Fix python dependencies in ParseDocument # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12619) Unable to instantiate ParseDocument processor
[
https://issues.apache.org/jira/browse/NIFI-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12619:
--
Status: Patch Available (was: Open)
> Unable to instantiate ParseDocument processor
> -
>
> Key: NIFI-12619
> URL: https://issues.apache.org/jira/browse/NIFI-12619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M1
> Environment: Python 3.11.6
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> Trying to instantiate the Python processor ParseDocument, I get this error:
>
> {code:java}
> 2024-01-16 18:00:00,573 INFO org.apache.nifi.py4j.ExtensionManager Importing
> dependencies ['langchain', 'unstructured', 'unstructured-inference',
> 'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
> 'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'] for
> ParseDocument to
> /Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT
> using command
> ['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
> '-m', 'pip', 'install', '--no-cache-dir', '--target',
> '/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
> 'langchain', 'unstructured', 'unstructured-inference',
> 'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
> 'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx']
> 2024-01-16 18:00:15,752 ERROR py4j.java_gateway There was an exception while
> executing the Python Proxy on the Python Side.
> Traceback (most recent call last):
> File
> "/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/py4j/java_gateway.py",
> line 2466, in _call_proxy
> return_value = getattr(self.pool[obj_id], method)(*params)
>^^^
> File
> "/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./python/framework/Controller.py",
> line 72, in downloadDependencies
> self.extensionManager.import_external_dependencies(processor_details,
> work_dir)
> File
> "/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/ExtensionManager.py",
> line 511, in import_external_dependencies
> raise RuntimeError(f"Failed to import requirements for {class_name}:
> process exited with status code {result}")
> RuntimeError: Failed to import requirements for ParseDocument: process exited
> with status code
> CompletedProcess(args=['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
> '-m', 'pip', 'install', '--no-cache-dir', '--target',
> '/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
> 'langchain', 'unstructured', 'unstructured-inference',
> 'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
> 'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'],
> returncode=1) {code}
> If trying to run the pip command manually, I get
>
>
> {code:java}
> no matches found: pdfminer.six[image] {code}
> Changing the required dependency to just *pdfminer.six* fixes the issue and I
> can instantiate the processor.
>
> However when trying to use it against a PDF file, I get:
>
> {code:java}
> ModuleNotFoundError: No module named 'pikepdf'
> ModuleNotFoundError: No module named 'pypdf'{code}
> After adding the above dependencies, I get:
> {code:java}
> pdf2image.exceptions.PDFInfoNotInstalledError: Unable to get page count. Is
> poppler installed and in PATH? {code}
> Based on
> [https://pdf2image.readthedocs.io/en/latest/installation.html]
> It sounds like poppler would need to be installed separately. I did it with
> brew for my local instance. Probably worth adding this in the docs if doable.
> At this point I was able to use the processor to parse a PDF file.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12619) Unable to instantiate ParseDocument processor
Pierre Villard created NIFI-12619:
-
Summary: Unable to instantiate ParseDocument processor
Key: NIFI-12619
URL: https://issues.apache.org/jira/browse/NIFI-12619
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 2.0.0-M1
Environment: Python 3.11.6
Reporter: Pierre Villard
Assignee: Pierre Villard
Trying to instantiate the Python processor ParseDocument, I get this error:
{code:java}
2024-01-16 18:00:00,573 INFO org.apache.nifi.py4j.ExtensionManager Importing
dependencies ['langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'] for
ParseDocument to
/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT
using command
['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
'langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx']
2024-01-16 18:00:15,752 ERROR py4j.java_gateway There was an exception while
executing the Python Proxy on the Python Side.
Traceback (most recent call last):
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/py4j/java_gateway.py",
line 2466, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
^^^
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./python/framework/Controller.py",
line 72, in downloadDependencies
self.extensionManager.import_external_dependencies(processor_details,
work_dir)
File
"/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/python/framework/ExtensionManager.py",
line 511, in import_external_dependencies
raise RuntimeError(f"Failed to import requirements for {class_name}:
process exited with status code {result}")
RuntimeError: Failed to import requirements for ParseDocument: process exited
with status code
CompletedProcess(args=['/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/Users/pierre/dev/github/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/./work/python/extensions/ParseDocument/2.0.0-SNAPSHOT',
'langchain', 'unstructured', 'unstructured-inference',
'unstructured_pytesseract', 'numpy', 'opencv-python', 'pdf2image',
'pdfminer.six[image]', 'python-docx', 'openpyxl', 'python-pptx'], returncode=1)
{code}
If trying to run the pip command manually, I get
{code:java}
no matches found: pdfminer.six[image] {code}
Changing the required dependency to just *pdfminer.six* fixes the issue and I
can instantiate the processor.
However when trying to use it against a PDF file, I get:
{code:java}
ModuleNotFoundError: No module named 'pikepdf'
ModuleNotFoundError: No module named 'pypdf'{code}
After adding the above dependencies, I get:
{code:java}
pdf2image.exceptions.PDFInfoNotInstalledError: Unable to get page count. Is
poppler installed and in PATH? {code}
Based on
[https://pdf2image.readthedocs.io/en/latest/installation.html]
It sounds like poppler would need to be installed separately. I did it with
brew for my local instance. Probably worth adding this in the docs if doable.
At this point I was able to use the processor to parse a PDF file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12604: Empty Queue [nifi]
mcgilman commented on PR #8246: URL: https://github.com/apache/nifi/pull/8246#issuecomment-1894317315 Thanks for the review @rfellows! Please see my latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12554 Placed all Jolt related code under a new nifi-jolt-bundle and refactored to reduce duplicate code. [nifi]
dan-s1 commented on PR #8249: URL: https://github.com/apache/nifi/pull/8249#issuecomment-1894290354 @exceptionfactory Thanks for clarifying what the builds were failing on as I could not see that. I only saw Error: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12618 Upgrade Azure SDK BOM from 1.2.18 to 1.2.19 [nifi]
joewitt commented on PR #8254: URL: https://github.com/apache/nifi/pull/8254#issuecomment-1894234053 +1 thanks! please self merge from here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12617) Change the default nifi.web.https.host value from 127.0.0.1 to a host name
[
https://issues.apache.org/jira/browse/NIFI-12617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807371#comment-17807371
]
David Handermann commented on NIFI-12617:
-
Thanks for the suggestion [~fgerlits], this sounds like a reasonable
improvement from an initial introduction perspective. Are you planning on
submitting a pull request?
> Change the default nifi.web.https.host value from 127.0.0.1 to a host name
> --
>
> Key: NIFI-12617
> URL: https://issues.apache.org/jira/browse/NIFI-12617
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Priority: Minor
>
> As of NiFi 2.0.0-M1, it is no longer possible to connect to NiFi on
> {{{}[https://127.0.0.1:8443|https://127.0.0.1:8443/]{}}}, but the default
> {{nifi.properties}} file still contains
> {noformat}
> nifi.web.https.host=127.0.0.1{noformat}
> and (therefore?) the log file still says
> {noformat}
> 2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer
> NiFi has started. The UI is available at the following URLs:
> 2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer
> https://127.0.0.1:8443/nifi{noformat}
> The default could be changed from {{127.0.0.1}} to (for example)
> {{{}localhost{}}}, to make the life of novice users easier, and to reduce the
> number of complaints coming from us.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12604) Add support to empty queues
[ https://issues.apache.org/jira/browse/NIFI-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807367#comment-17807367 ] Mark Bean commented on NIFI-12604: -- Thanks [~joewitt] , [~mcgilman] . Makes more sense now. > Add support to empty queues > --- > > Key: NIFI-12604 > URL: https://issues.apache.org/jira/browse/NIFI-12604 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Introduce actions for emptying a queue or all queues within a Process Group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-9458 Replace SimpleDateFormat with DateTimeFormatter [nifi]
exceptionfactory commented on PR #8248: URL: https://github.com/apache/nifi/pull/8248#issuecomment-1894206758 > @exceptionfactory I took a quick "high level" peek at this. There are lot of changed files, but I searched for any which might have been missed inadvertently. I found instances of SimpleDateFormat usage in the following: > > ``` > ./nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java > ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ParseCEF.java > ``` Thanks for pointing out those specific examples @markobean. Those are the two remaining direct references to `SimpleDateFormat` because both components make direct use of the Jackson `ObjectMapper`, which requires an instance of `DateFormat` to drive formatting. Unfortunately those references cannot be changed. The key changes from a NiFi usage perspective are in `nifi-record` and `nifi-record-path`, where first-class use of `SimpleDateFormat` has been changed. There are many other changes in test classes, and the primary motivation for those changes was to provide better baseline examples for future development. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-9458 Replace SimpleDateFormat with DateTimeFormatter [nifi]
markobean commented on PR #8248: URL: https://github.com/apache/nifi/pull/8248#issuecomment-1894182219 @exceptionfactory I took a quick "high level" peek at this. There are lot of changed files, but I searched for any which might have been missed inadvertently. I found instances of SimpleDateFormat usage in the following: ``` ./nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/AbstractMongoProcessor.java ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ParseCEF.java ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12604) Add support to empty queues
[ https://issues.apache.org/jira/browse/NIFI-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807357#comment-17807357 ] Matt Gilman commented on NIFI-12604: [~markbean] It was becoming more and more challenging to keep dependencies updated and eliminate those with published CVEs. Some of the dependencies are not supported or actively maintained. Please see NIFI-11481 for the initial motivation. > Add support to empty queues > --- > > Key: NIFI-12604 > URL: https://issues.apache.org/jira/browse/NIFI-12604 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Introduce actions for emptying a queue or all queues within a Process Group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12604: Empty Queue [nifi]
rfellows commented on code in PR #8246:
URL: https://github.com/apache/nifi/pull/8246#discussion_r1453611366
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/queue/queue.effects.ts:
##
@@ -0,0 +1,342 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Injectable } from '@angular/core';
+import { Actions, createEffect, ofType } from '@ngrx/effects';
+import * as QueueActions from './queue.actions';
+import { Store } from '@ngrx/store';
+import {
+asyncScheduler,
+catchError,
+filter,
+from,
+interval,
+map,
+of,
+switchMap,
+take,
+takeUntil,
+tap,
+withLatestFrom
+} from 'rxjs';
+import { selectDropConnectionId, selectDropProcessGroupId,
selectDropRequestEntity } from './queue.selectors';
+import { QueueService } from '../../service/queue.service';
+import { DropRequest } from './index';
+import { CancelDialog } from
'../../../../ui/common/cancel-dialog/cancel-dialog.component';
+import { MatDialog } from '@angular/material/dialog';
+import { NiFiCommon } from '../../../../service/nifi-common.service';
+import { isDefinedAndNotNull } from '../../../../state/shared';
+import { YesNoDialog } from
'../../../../ui/common/yes-no-dialog/yes-no-dialog.component';
+import { OkDialog } from '../../../../ui/common/ok-dialog/ok-dialog.component';
+import { loadConnection, loadProcessGroup } from '../flow/flow.actions';
+import { resetQueueState } from './queue.actions';
+
+@Injectable()
+export class QueueEffects {
+constructor(
+private actions$: Actions,
+private store: Store,
+private queueService: QueueService,
+private dialog: MatDialog,
+private nifiCommon: NiFiCommon
+) {}
+
+promptEmptyQueueRequest$ = createEffect(
+() =>
+this.actions$.pipe(
+ofType(QueueActions.promptEmptyQueueRequest),
+map((action) => action.request),
+tap((request) => {
+const dialogReference = this.dialog.open(YesNoDialog, {
+data: {
+title: 'Empty Queue',
+message:
+'Are you sure you want to empty this queue?
All FlowFiles waiting at the time of the request will be removed.'
+},
+panelClass: 'small-dialog'
+});
+
+
dialogReference.componentInstance.yes.pipe(take(1)).subscribe(() => {
+this.store.dispatch(
+QueueActions.submitEmptyQueueRequest({
+request
+})
+);
+});
+})
+),
+{ dispatch: false }
+);
+
+submitEmptyQueueRequest$ = createEffect(() =>
+this.actions$.pipe(
+ofType(QueueActions.submitEmptyQueueRequest),
+map((action) => action.request),
+switchMap((request) => {
+const dialogReference = this.dialog.open(CancelDialog, {
+data: {
+title: 'Empty Queue',
+message: 'Waiting for queue to empty...'
+},
+disableClose: true,
+panelClass: 'small-dialog'
+});
+
+
dialogReference.componentInstance.cancel.pipe(take(1)).subscribe(() => {
+
this.store.dispatch(QueueActions.stopPollingEmptyQueueRequest());
+});
+
+return
from(this.queueService.submitEmptyQueueRequest(request)).pipe(
+map((response) =>
+QueueActions.submitEmptyQueueRequestSuccess({
+response: {
+dropEntity: response
+}
+})
+),
+catchError((error) =>
+of(
+QueueActions.queueApiError({
+
[jira] [Commented] (NIFI-12604) Add support to empty queues
[ https://issues.apache.org/jira/browse/NIFI-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807349#comment-17807349 ] Joe Witt commented on NIFI-12604: - [~markbean]Take a look at the epic/parent task this is part of https://issues.apache.org/jira/browse/NIFI-12400. This is the new UI implementation > Add support to empty queues > --- > > Key: NIFI-12604 > URL: https://issues.apache.org/jira/browse/NIFI-12604 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Introduce actions for emptying a queue or all queues within a Process Group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12604) Add support to empty queues
[ https://issues.apache.org/jira/browse/NIFI-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807348#comment-17807348 ] Mark Bean commented on NIFI-12604: -- [~mcgilman] Can you explain what this issue is adding? There already exists the ability empty queues at a Process Group level. Thanks. > Add support to empty queues > --- > > Key: NIFI-12604 > URL: https://issues.apache.org/jira/browse/NIFI-12604 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Introduce actions for emptying a queue or all queues within a Process Group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12618) Upgrade Azure SDK BOM to 1.2.19
[ https://issues.apache.org/jira/browse/NIFI-12618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12618: Status: Patch Available (was: Open) > Upgrade Azure SDK BOM to 1.2.19 > --- > > Key: NIFI-12618 > URL: https://issues.apache.org/jira/browse/NIFI-12618 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 10m > Remaining Estimate: 0h > > The Azure SDK BOM should be upgraded from 1.2.18 to 1.2.19 to incorporate > incremental updates to Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12618 Upgrade Azure SDK BOM from 1.2.18 to 1.2.19 [nifi]
exceptionfactory opened a new pull request, #8254: URL: https://github.com/apache/nifi/pull/8254 # Summary [NIFI-12618](https://issues.apache.org/jira/browse/NIFI-12618) Upgrades Azure SDK Bill of Materials dependencies from 1.2.18 to 1.2.19, incorporating the latest Azure SDK libraries for various components. Additional changes include upgrading MSAL4J from 1.14.0 to 1.14.2 and adding CVE-2023-36052 to the list of suppressed findings for the OWASP Plugin as the vulnerability applies to the Azure CLI and not Azure Java packages. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12618) Upgrade Azure SDK BOM to 1.2.19
David Handermann created NIFI-12618: --- Summary: Upgrade Azure SDK BOM to 1.2.19 Key: NIFI-12618 URL: https://issues.apache.org/jira/browse/NIFI-12618 Project: Apache NiFi Issue Type: Improvement Reporter: David Handermann Assignee: David Handermann Fix For: 1.25.0, 2.0.0-M2 The Azure SDK BOM should be upgraded from 1.2.18 to 1.2.19 to incorporate incremental updates to Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11389) Controller Services's link to referencing Controller Services components not always working
[ https://issues.apache.org/jira/browse/NIFI-11389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-11389: --- Fix Version/s: 1.25.0 2.0.0-M2 Resolution: Fixed Status: Resolved (was: Patch Available) > Controller Services's link to referencing Controller Services components not > always working > --- > > Key: NIFI-11389 > URL: https://issues.apache.org/jira/browse/NIFI-11389 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M1, 1.20.0, 1.19.1 >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 50m > Remaining Estimate: 0h > > Controller Services' Setting tab has a _Referencing Components_ section on > the right side. > There are hyperlinks to the Processors and/or Controller Services which > reference this controller service. These links do not always work when > referencing controller services. > To reproduce: > Create StandardRestrictedSSLContextService on top level nifi graph (i.e. > right click top level grid -> Configure -> Controller Services) > Create Process Group > Inside Process Group create DistributedMapCacheServer > and point _SSL Context Service_ to StandardRestrictedSSLContextService just > created > Go to StandardRestrictedSSLContextService's Setting's tab and verify > DistributedMapCacheServer appears in the _Referencing Components_ section. > Clicking on the link to DistributedMapCacheServer will not take us to that > controller service. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11389) Controller Services's link to referencing Controller Services components not always working
[ https://issues.apache.org/jira/browse/NIFI-11389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807340#comment-17807340 ] ASF subversion and git services commented on NIFI-11389: Commit 3b5d729a3a7b282a5ba2f4a9229f1de04078aa28 in nifi's branch refs/heads/main from NissimShiman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3b5d729a3a ] NIFI-11389 Fixed controller services's link to referencing controller (#8099) services of lesser scope This closes #8099 > Controller Services's link to referencing Controller Services components not > always working > --- > > Key: NIFI-11389 > URL: https://issues.apache.org/jira/browse/NIFI-11389 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M1, 1.20.0, 1.19.1 >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > Labels: backport-needed > Time Spent: 40m > Remaining Estimate: 0h > > Controller Services' Setting tab has a _Referencing Components_ section on > the right side. > There are hyperlinks to the Processors and/or Controller Services which > reference this controller service. These links do not always work when > referencing controller services. > To reproduce: > Create StandardRestrictedSSLContextService on top level nifi graph (i.e. > right click top level grid -> Configure -> Controller Services) > Create Process Group > Inside Process Group create DistributedMapCacheServer > and point _SSL Context Service_ to StandardRestrictedSSLContextService just > created > Go to StandardRestrictedSSLContextService's Setting's tab and verify > DistributedMapCacheServer appears in the _Referencing Components_ section. > Clicking on the link to DistributedMapCacheServer will not take us to that > controller service. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11389) Controller Services's link to referencing Controller Services components not always working
[ https://issues.apache.org/jira/browse/NIFI-11389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807341#comment-17807341 ] ASF subversion and git services commented on NIFI-11389: Commit 45713f6eec82e0a4bcc2aabd965bbf3f5c541c46 in nifi's branch refs/heads/support/nifi-1.x from NissimShiman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=45713f6eec ] NIFI-11389 Fixed controller services's link to referencing controller (#8194) services of lesser scope This closes #8194 > Controller Services's link to referencing Controller Services components not > always working > --- > > Key: NIFI-11389 > URL: https://issues.apache.org/jira/browse/NIFI-11389 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M1, 1.20.0, 1.19.1 >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > Labels: backport-needed > Time Spent: 50m > Remaining Estimate: 0h > > Controller Services' Setting tab has a _Referencing Components_ section on > the right side. > There are hyperlinks to the Processors and/or Controller Services which > reference this controller service. These links do not always work when > referencing controller services. > To reproduce: > Create StandardRestrictedSSLContextService on top level nifi graph (i.e. > right click top level grid -> Configure -> Controller Services) > Create Process Group > Inside Process Group create DistributedMapCacheServer > and point _SSL Context Service_ to StandardRestrictedSSLContextService just > created > Go to StandardRestrictedSSLContextService's Setting's tab and verify > DistributedMapCacheServer appears in the _Referencing Components_ section. > Clicking on the link to DistributedMapCacheServer will not take us to that > controller service. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11389 Fixed controller services's link to referencing controller services of lesser scope - 1.x branch [nifi]
mcgilman merged PR #8194: URL: https://github.com/apache/nifi/pull/8194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11389 Fixed controller service's link to referencing controller services of lesser scope [nifi]
mcgilman merged PR #8099: URL: https://github.com/apache/nifi/pull/8099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12506 /nifi-api/flow/metrics endpoint times out if flow is big [nifi]
exceptionfactory commented on code in PR #8251:
URL: https://github.com/apache/nifi/pull/8251#discussion_r1453657239
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##
@@ -6117,27 +6123,53 @@ protected Collection
populateFlowMetrics() {
// Get Connection Status Analytics (predictions, e.g.)
Set connections =
controllerFacade.getFlowManager().findAllConnections();
-for (Connection c : connections) {
-// If a ResourceNotFoundException is thrown, analytics hasn't been
enabled
+Collection> predictions =
Collections.synchronizedList(new ArrayList<>());
+
+final boolean analyticsEnabled =
Boolean.parseBoolean(properties.getProperty(NiFiProperties.ANALYTICS_PREDICTION_ENABLED,
Boolean.FALSE.toString()));
+
+if (analyticsEnabled) {
+if (parallelProcessingThreadPool == null) {
+parallelProcessingThreadPool =
createParallelProcessingThreadPool();
Review Comment:
See same note on the pull request for the main branch. In this case as well,
the thread pool should be created when the ServiceFacade class is initialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12506 /nifi-api/flow/metrics endpoint times out if flow is big [nifi]
exceptionfactory commented on code in PR #8158:
URL: https://github.com/apache/nifi/pull/8158#discussion_r1453623270
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##
@@ -6233,27 +6239,54 @@ protected Collection
populateFlowMetrics() {
// Get Connection Status Analytics (predictions, e.g.)
Set connections =
controllerFacade.getFlowManager().findAllConnections();
-for (Connection c : connections) {
-// If a ResourceNotFoundException is thrown, analytics hasn't been
enabled
+Collection> predictions =
Collections.synchronizedList(new ArrayList<>());
+final boolean analyticsEnabled =
Boolean.parseBoolean(properties.getProperty(NiFiProperties.ANALYTICS_PREDICTION_ENABLED,
Boolean.FALSE.toString()));
+
+if (analyticsEnabled) {
+if (parallelProcessingExecutor == null) {
+parallelProcessingExecutor =
Executors.newVirtualThreadPerTaskExecutor();
+}
Review Comment:
This conditional instantiation is not thread safe for multiple invocations
of the same method. The property needs to be checked earlier in the process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-11389 Fixed controller service's link to referencing controller services of lesser scope [nifi]
mcgilman commented on PR #8099: URL: https://github.com/apache/nifi/pull/8099#issuecomment-1894011592 Will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12441) Add No Tracking Strategy to ListS3
[ https://issues.apache.org/jira/browse/NIFI-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12441: Fix Version/s: 2.0.0-M2 Resolution: Fixed Status: Resolved (was: Patch Available) > Add No Tracking Strategy to ListS3 > -- > > Key: NIFI-12441 > URL: https://issues.apache.org/jira/browse/NIFI-12441 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Julien G. >Assignee: Julien G. >Priority: Major > Fix For: 2.0.0-M2 > > Time Spent: 50m > Remaining Estimate: 0h > > As was done for ListGCS in > [NIFI-11891|https://issues.apache.org/jira/browse/NIFI-11891], I want to make > it possible to have no state in ListS3 so that I can periodically retrieve > all my objects in S3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12441 Added No Tracking listing strategy to ListS3 [nifi]
exceptionfactory closed pull request #8088: NIFI-12441 Added No Tracking listing strategy to ListS3 URL: https://github.com/apache/nifi/pull/8088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12441) Add No Tracking Strategy to ListS3
[ https://issues.apache.org/jira/browse/NIFI-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807307#comment-17807307 ] ASF subversion and git services commented on NIFI-12441: Commit 9a919339090b26b0b39321758b8f8074597320ca in nifi's branch refs/heads/main from Juldrixx [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9a91933909 ] NIFI-12441 Added No Tracking Strategy to ListS3 This closes #8088 Signed-off-by: David Handermann > Add No Tracking Strategy to ListS3 > -- > > Key: NIFI-12441 > URL: https://issues.apache.org/jira/browse/NIFI-12441 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Julien G. >Assignee: Julien G. >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > As was done for ListGCS in > [NIFI-11891|https://issues.apache.org/jira/browse/NIFI-11891], I want to make > it possible to have no state in ListS3 so that I can periodically retrieve > all my objects in S3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12593) ValidateCSV - get all constraint violations for an invalid line
[ https://issues.apache.org/jira/browse/NIFI-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12593: Resolution: Fixed Status: Resolved (was: Patch Available) > ValidateCSV - get all constraint violations for an invalid line > --- > > Key: NIFI-12593 > URL: https://issues.apache.org/jira/browse/NIFI-12593 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Right now the ValidateCSV will invalidate a line as soon as a column is > violating the specified constraint and will go the next one. It means that > for an invalid line, we'll only know about the first violation but the line > may be invalid for many columns. > It'd be nice to have the option to get all of the violations for a given > line. This should be optional, and false by default, as this would impact the > performances of the processing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12593) ValidateCSV - get all constraint violations for an invalid line
[ https://issues.apache.org/jira/browse/NIFI-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807302#comment-17807302 ] ASF subversion and git services commented on NIFI-12593: Commit 66e6eb55a9334e34db7575230a2d6499839a3ca2 in nifi's branch refs/heads/support/nifi-1.x from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=66e6eb55a9 ] NIFI-12593 Added Include all violations property to ValidateCsv This closes #8229 Signed-off-by: David Handermann (cherry picked from commit dd45e7edcb3cc1fa3385bec70cac075fbf60acd7) > ValidateCSV - get all constraint violations for an invalid line > --- > > Key: NIFI-12593 > URL: https://issues.apache.org/jira/browse/NIFI-12593 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Right now the ValidateCSV will invalidate a line as soon as a column is > violating the specified constraint and will go the next one. It means that > for an invalid line, we'll only know about the first violation but the line > may be invalid for many columns. > It'd be nice to have the option to get all of the violations for a given > line. This should be optional, and false by default, as this would impact the > performances of the processing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12616: Added some Use Case docs for Python processors and update… [nifi]
markap14 opened a new pull request, #8253: URL: https://github.com/apache/nifi/pull/8253 …d Runtime Manifests to include Python based processors as well as Use Case/MultiProcessorUseCase documentation elements. Refactored/cleaned up some of the Python code and added unit tests. # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12593) ValidateCSV - get all constraint violations for an invalid line
[ https://issues.apache.org/jira/browse/NIFI-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807300#comment-17807300 ] ASF subversion and git services commented on NIFI-12593: Commit dd45e7edcb3cc1fa3385bec70cac075fbf60acd7 in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dd45e7edcb ] NIFI-12593 Added Include all violations property to ValidateCsv This closes #8229 Signed-off-by: David Handermann > ValidateCSV - get all constraint violations for an invalid line > --- > > Key: NIFI-12593 > URL: https://issues.apache.org/jira/browse/NIFI-12593 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Right now the ValidateCSV will invalidate a line as soon as a column is > violating the specified constraint and will go the next one. It means that > for an invalid line, we'll only know about the first violation but the line > may be invalid for many columns. > It'd be nice to have the option to get all of the violations for a given > line. This should be optional, and false by default, as this would impact the > performances of the processing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12593 - ValidateCSV - get all constraint violations for an invalid line [nifi]
exceptionfactory closed pull request #8229: NIFI-12593 - ValidateCSV - get all constraint violations for an invalid line URL: https://github.com/apache/nifi/pull/8229 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
juldrixx commented on code in PR #8240:
URL: https://github.com/apache/nifi/pull/8240#discussion_r1453543972
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -398,6 +401,7 @@ public class NiFiProperties extends ApplicationProperties {
private static final String DEFAULT_SECURITY_USER_JWS_KEY_ROTATION_PERIOD
= "PT1H";
public static final String DEFAULT_WEB_SHOULD_SEND_SERVER_VERSION = "true";
public static final int DEFAULT_LISTENER_BOOTSTRAP_PORT = 0;
+public static final String
DEFAULT_CLUSTER_LEADER_ELECTION_KUBERNETES_LEASE_PREFIX = null;
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12590 Add options to add a prefix on Kubernetes resources for Cl… [nifi]
exceptionfactory commented on code in PR #8240:
URL: https://github.com/apache/nifi/pull/8240#discussion_r1453528762
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -398,6 +401,7 @@ public class NiFiProperties extends ApplicationProperties {
private static final String DEFAULT_SECURITY_USER_JWS_KEY_ROTATION_PERIOD
= "PT1H";
public static final String DEFAULT_WEB_SHOULD_SEND_SERVER_VERSION = "true";
public static final int DEFAULT_LISTENER_BOOTSTRAP_PORT = 0;
+public static final String
DEFAULT_CLUSTER_LEADER_ELECTION_KUBERNETES_LEASE_PREFIX = null;
Review Comment:
Since this default value is `null`, it does not provide much value, so it
can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2229 Encrypt sensitive properties (bugfix) [nifi-minifi-cpp]
fgerlits commented on code in PR #1717:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1717#discussion_r1453525509
##
libminifi/test/unit/JsonFlowSerializerTests.cpp:
##
@@ -0,0 +1,425 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+
+#include "../Catch.h"
+#include "../ConfigurationTestController.h"
+#include "core/flow/FlowSchema.h"
+#include "core/json/JsonFlowSerializer.h"
+#include "core/json/JsonNode.h"
+#include "utils/crypto/EncryptionProvider.h"
+#include "utils/crypto/property_encryption/PropertyEncryptionUtils.h"
+#include "utils/StringUtils.h"
+
+namespace core = org::apache::nifi::minifi::core;
+namespace utils = org::apache::nifi::minifi::utils;
+
+constexpr std::string_view config_json = R"({
+"encodingVersion": {
+"majorVersion": 2,
+"minorVersion": 0
+},
+"maxTimerDrivenThreadCount": 1,
+"maxEventDrivenThreadCount": 1,
+"parameterContexts": [],
+"rootGroup": {
+"identifier": "8b4d66dc-9085-4722-b35b-3492f363baa3",
+"instanceIdentifier": "0fab8422-3fbe-45e0-bd3a-324f5cb0592b",
+"name": "root",
+"position": {
+"x": 0.0,
+"y": 0.0
+},
+"processGroups": [],
+"remoteProcessGroups": [],
+"processors": [
+{
+"identifier": "aabb6d26-8a8d-4338-92c9-1b8c67ec18e0",
+"instanceIdentifier": "ebbb1400-9fe6-4566-affd-1484a1cee58f",
+"name": "GenerateFlowFile",
+"comments": "",
+"position": {
+"x": 687.0,
+"y": 372.0
+},
+"type": "org.apache.nifi.minifi.processors.GenerateFlowFile",
+"bundle": {
+"group": "org.apache.nifi.minifi",
+"artifact": "minifi-standard-processors",
+"version": "1.23.06"
+},
+"properties": {
+"File Size": "100B",
+"Batch Size": "1",
+"Unique FlowFiles": "true",
+"Data Format": "Text"
+},
+"propertyDescriptors": {
+"File Size": {
+"name": "File Size",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Batch Size": {
+"name": "Batch Size",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Unique FlowFiles": {
+"name": "Unique FlowFiles",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Custom Text": {
+"name": "Custom Text",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Data Format": {
+"name": "Data Format",
+"identifiesControllerService": false,
+"sensitive": false
+}
+},
+"style": {},
+"schedulingPeriod": "15 sec",
+"schedulingStrategy": "TIMER_DRIVEN",
+"executionNode": "ALL",
+"penaltyDuration": "3 ms",
+"yieldDuration": "1000 ms",
+"bulletinLevel": "WARN",
+"runDurationMillis": 0,
+"concurrentlySchedulableTaskCount": 1,
+"autoTerminatedRelationships": [],
+"componentType": "PROCESSOR",
+"groupIdentifier": "8b4d66dc-9085-4722-b35b-3492f363baa3"
+},
+{
+"identifier": "469617f1-3898-4bbf-91fe-27d8f4dd2a75",
+"instanceIdentifier": "a43e3d65-bff9-4557-899b-43f680e4b533",
+"name": "InvokeHTTP",
+"comments": "",
+"position": {
+"x": 1031.0,
+
[jira] [Assigned] (MINIFICPP-2288) LoggerConfiguration::loggers gets spammed with unused loggers
[ https://issues.apache.org/jira/browse/MINIFICPP-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink reassigned MINIFICPP-2288: -- Assignee: Martin Zink > LoggerConfiguration::loggers gets spammed with unused loggers > - > > Key: MINIFICPP-2288 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2288 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Blocker > > LoggerConfiguration caches shared_ptr for every logger ever created. This > could cause to very high memory usage, with no real upside. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12617) Change the default nifi.web.https.host value from 127.0.0.1 to a host name
Ferenc Gerlits created NIFI-12617:
-
Summary: Change the default nifi.web.https.host value from
127.0.0.1 to a host name
Key: NIFI-12617
URL: https://issues.apache.org/jira/browse/NIFI-12617
Project: Apache NiFi
Issue Type: Improvement
Reporter: Ferenc Gerlits
As of NiFi 2.0.0-M1, it is no longer possible to connect to NiFi on
{{{}[https://127.0.0.1:8443|https://127.0.0.1:8443/]{}}}, but the default
{{nifi.properties}} file still contains
{noformat}
nifi.web.https.host=127.0.0.1{noformat}
and (therefore?) the log file still says
{noformat}
2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer NiFi
has started. The UI is available at the following URLs:
2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer
https://127.0.0.1:8443/nifi{noformat}
The default could be changed from {{127.0.0.1}} to (for example)
{{{}localhost{}}}, to make the life of novice users easier, and to reduce the
number of complaints coming from us.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12615) BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page.
[ https://issues.apache.org/jira/browse/NIFI-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12615: --- Description: We've seen this same error regarding tables where we are trying to support initial sorting for a table and then leverage the even/odd variables to class the rows. The solution is to implement our own sorting and not use the MatSort. Also, as part of this we should update other cases where we rely on MatSort/sortingDataAccessor do do our sorting. was:We've seen this same error regarding tables where we are trying to support initial sorting for a table and then leverage the even/odd variables to class the rows. The solution is to implement our own sorting and not use the MatSort. > BUG: ExpressionChangedAfterItHasBeenCheckedError on the Counters page. > -- > > Key: NIFI-12615 > URL: https://issues.apache.org/jira/browse/NIFI-12615 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Attachments: Screenshot 2024-01-16 at 8.47.12 AM.png > > Time Spent: 20m > Remaining Estimate: 0h > > We've seen this same error regarding tables where we are trying to support > initial sorting for a table and then leverage the even/odd variables to class > the rows. The solution is to implement our own sorting and not use the > MatSort. > Also, as part of this we should update other cases where we rely on > MatSort/sortingDataAccessor do do our sorting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12615] - fix ExpressionChanged error on Counters page. [nifi]
mcgilman commented on PR #8252: URL: https://github.com/apache/nifi/pull/8252#issuecomment-1893831968 Will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2229 Encrypt sensitive properties (bugfix) [nifi-minifi-cpp]
lordgamez commented on code in PR #1717:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1717#discussion_r1453484282
##
libminifi/test/unit/JsonFlowSerializerTests.cpp:
##
@@ -0,0 +1,425 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+
+#include "../Catch.h"
+#include "../ConfigurationTestController.h"
+#include "core/flow/FlowSchema.h"
+#include "core/json/JsonFlowSerializer.h"
+#include "core/json/JsonNode.h"
+#include "utils/crypto/EncryptionProvider.h"
+#include "utils/crypto/property_encryption/PropertyEncryptionUtils.h"
+#include "utils/StringUtils.h"
+
+namespace core = org::apache::nifi::minifi::core;
+namespace utils = org::apache::nifi::minifi::utils;
+
+constexpr std::string_view config_json = R"({
+"encodingVersion": {
+"majorVersion": 2,
+"minorVersion": 0
+},
+"maxTimerDrivenThreadCount": 1,
+"maxEventDrivenThreadCount": 1,
+"parameterContexts": [],
+"rootGroup": {
+"identifier": "8b4d66dc-9085-4722-b35b-3492f363baa3",
+"instanceIdentifier": "0fab8422-3fbe-45e0-bd3a-324f5cb0592b",
+"name": "root",
+"position": {
+"x": 0.0,
+"y": 0.0
+},
+"processGroups": [],
+"remoteProcessGroups": [],
+"processors": [
+{
+"identifier": "aabb6d26-8a8d-4338-92c9-1b8c67ec18e0",
+"instanceIdentifier": "ebbb1400-9fe6-4566-affd-1484a1cee58f",
+"name": "GenerateFlowFile",
+"comments": "",
+"position": {
+"x": 687.0,
+"y": 372.0
+},
+"type": "org.apache.nifi.minifi.processors.GenerateFlowFile",
+"bundle": {
+"group": "org.apache.nifi.minifi",
+"artifact": "minifi-standard-processors",
+"version": "1.23.06"
+},
+"properties": {
+"File Size": "100B",
+"Batch Size": "1",
+"Unique FlowFiles": "true",
+"Data Format": "Text"
+},
+"propertyDescriptors": {
+"File Size": {
+"name": "File Size",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Batch Size": {
+"name": "Batch Size",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Unique FlowFiles": {
+"name": "Unique FlowFiles",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Custom Text": {
+"name": "Custom Text",
+"identifiesControllerService": false,
+"sensitive": false
+},
+"Data Format": {
+"name": "Data Format",
+"identifiesControllerService": false,
+"sensitive": false
+}
+},
+"style": {},
+"schedulingPeriod": "15 sec",
+"schedulingStrategy": "TIMER_DRIVEN",
+"executionNode": "ALL",
+"penaltyDuration": "3 ms",
+"yieldDuration": "1000 ms",
+"bulletinLevel": "WARN",
+"runDurationMillis": 0,
+"concurrentlySchedulableTaskCount": 1,
+"autoTerminatedRelationships": [],
+"componentType": "PROCESSOR",
+"groupIdentifier": "8b4d66dc-9085-4722-b35b-3492f363baa3"
+},
+{
+"identifier": "469617f1-3898-4bbf-91fe-27d8f4dd2a75",
+"instanceIdentifier": "a43e3d65-bff9-4557-899b-43f680e4b533",
+"name": "InvokeHTTP",
+"comments": "",
+"position": {
+"x": 1031.0,
+
[PR] [NIFI-12615] - fix ExpressionChanged error on Counters page. [nifi]
rfellows opened a new pull request, #8252: URL: https://github.com/apache/nifi/pull/8252 [NIFI-12615](https://issues.apache.org/jira/browse/NIFI-12615) Fixed the `EpressionChangedAfterItHasBeenCheckedError` on the Counters page by implementing our own sorting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12506 /nifi-api/flow/metrics endpoint times out if flow is big [nifi]
timeabarna commented on PR #8158: URL: https://github.com/apache/nifi/pull/8158#issuecomment-1893818027 Thanks @exceptionfactory for your review. I've updated this PR with Virtual Threads and opened [this PR ](https://github.com/apache/nifi/pull/8251) for the support branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12616) Enable @use_case and @multi_processor_use_case decorators to be added to Python Processors
Mark Payne created NIFI-12616: - Summary: Enable @use_case and @multi_processor_use_case decorators to be added to Python Processors Key: NIFI-12616 URL: https://issues.apache.org/jira/browse/NIFI-12616 Project: Apache NiFi Issue Type: Bug Components: Core Framework, Extensions Reporter: Mark Payne Assignee: Mark Payne Fix For: 2.0.0-M2 Currently, Python processors have no way of articulating specific use cases and multi-processor use cases in their docs. Introduce new decorators to allow for these. We use decorators here in order to keep the structure similar to that of Java but also because it offers a clean mechanism for defining the MultiProcessorUseCase, which becomes awkward if trying to include in the ProcessorDetails inner class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2262 Add processor to push logs to Grafana Loki through gRPC [nifi-minifi-cpp]
lordgamez commented on code in PR #1698:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1698#discussion_r1453461616
##
extensions/grafana-loki/PushGrafanaLoki.cpp:
##
@@ -0,0 +1,249 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "PushGrafanaLoki.h"
+
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "utils/ProcessorConfigUtils.h"
+#include "utils/StringUtils.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+void PushGrafanaLoki::LogBatch::add(const std::shared_ptr&
flowfile) {
+ gsl_Expects(state_manager_);
+ if (log_line_batch_wait_ && batched_flowfiles_.empty()) {
+start_push_time_ = std::chrono::steady_clock::now();
+std::unordered_map state;
+state["start_push_time"] =
std::to_string(std::chrono::duration_cast(start_push_time_.time_since_epoch()).count());
+logger_->log_debug("Saved start push time to state: {}",
state["start_push_time"]);
+state_manager_->set(state);
+ }
+ batched_flowfiles_.push_back(flowfile);
+}
+
+void PushGrafanaLoki::LogBatch::restore(const std::shared_ptr&
flowfile) {
+ batched_flowfiles_.push_back(flowfile);
+}
+
+std::vector>
PushGrafanaLoki::LogBatch::flush() {
+ gsl_Expects(state_manager_);
+ start_push_time_ = {};
+ auto result = batched_flowfiles_;
+ batched_flowfiles_.clear();
+ if (log_line_batch_wait_) {
+start_push_time_ = {};
+std::unordered_map state;
+logger_->log_debug("Reset start push time state");
+state["start_push_time"] = "0";
+state_manager_->set(state);
+ }
+ return result;
+}
+
+bool PushGrafanaLoki::LogBatch::isReady() const {
+ return (log_line_batch_size_ && batched_flowfiles_.size() >=
*log_line_batch_size_) || (log_line_batch_wait_ &&
std::chrono::steady_clock::now() - start_push_time_ >= *log_line_batch_wait_);
+}
+
+void PushGrafanaLoki::LogBatch::setLogLineBatchSize(std::optional
log_line_batch_size) {
+ log_line_batch_size_ = log_line_batch_size;
+}
+
+void
PushGrafanaLoki::LogBatch::setLogLineBatchWait(std::optional
log_line_batch_wait) {
+ log_line_batch_wait_ = log_line_batch_wait;
+}
+
+void PushGrafanaLoki::LogBatch::setStateManager(core::StateManager*
state_manager) {
+ state_manager_ = state_manager;
+}
+
+void
PushGrafanaLoki::LogBatch::setStartPushTime(std::chrono::steady_clock::time_point
start_push_time) {
+ start_push_time_ = start_push_time;
+}
+
+const core::Relationship PushGrafanaLoki::Self("__self__", "Marks the FlowFile
to be owned by this processor");
+
+std::shared_ptr
PushGrafanaLoki::getSSLContextService(core::ProcessContext& context) {
+ if (auto ssl_context =
context.getProperty(PushGrafanaLoki::SSLContextService)) {
+return
std::dynamic_pointer_cast(context.getControllerService(*ssl_context));
+ }
+ return std::shared_ptr{};
+}
+
+std::string PushGrafanaLoki::readLogLineFromFlowFile(const
std::shared_ptr& flow_file, core::ProcessSession& session) {
+ auto read_buffer_result = session.readBuffer(flow_file);
+ return {reinterpret_cast(read_buffer_result.buffer.data()),
read_buffer_result.buffer.size()};
+}
+
+void PushGrafanaLoki::setUpStateManager(core::ProcessContext& context) {
+ auto state_manager = context.getStateManager();
+ if (state_manager == nullptr) {
+throw Exception(PROCESSOR_EXCEPTION, "Failed to get StateManager");
+ }
+ log_batch_.setStateManager(state_manager);
+
+ std::unordered_map state_map;
+ if (state_manager->get(state_map)) {
+auto it = state_map.find("start_push_time");
+if (it != state_map.end()) {
+ logger_->log_info("Restored start push time from processor state: {}",
it->second);
+ std::chrono::steady_clock::time_point
start_push_time{std::chrono::milliseconds{std::stoll(it->second)}};
+ log_batch_.setStartPushTime(start_push_time);
+}
+ }
+}
+
+std::map
PushGrafanaLoki::buildStreamLabelMap(core::ProcessContext& context) {
+ std::map stream_label_map;
+ if (auto stream_labels_str = context.getProperty(StreamLabels)) {
+auto stream_labels =
utils::string::splitAndTrimRemovingEmpty(*stream_labels_str, ",");
+if (stream_labels.empty()) {
+ throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Missing or
