Re: [PR] NIFI-12331: add new unified PublishSlack Processor [nifi]
exceptionfactory commented on PR #8047: URL: https://github.com/apache/nifi/pull/8047#issuecomment-1839995418 Thanks for your work on this new Processor @sporadek, it provided helpful background for subsequent follow-on work in PR #8120. Thanks for the review as well @krisztina-zsihovszki. The other pull request is close to completion, so I am closing this PR in favor of the others. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12331: add new unified PublishSlack Processor [nifi]
exceptionfactory closed pull request #8047: NIFI-12331: add new unified PublishSlack Processor URL: https://github.com/apache/nifi/pull/8047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793125#comment-17793125
]
David Handermann commented on NIFI-12446:
-
Thanks for working on this [~EndzeitBegins]. I will leave this issue open for
now pending a new pull request for the main branch.
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Labels: backport-needed
> Fix For: 1.25.0
>
> Attachments: image-2023-11-30-16-18-43-040.png
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12446:
Fix Version/s: 1.25.0
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Labels: backport-needed
> Fix For: 1.25.0
>
> Attachments: image-2023-11-30-16-18-43-040.png
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12446:
Status: Patch Available (was: Open)
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Labels: backport-needed
> Attachments: image-2023-11-30-16-18-43-040.png
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793124#comment-17793124
]
ASF subversion and git services commented on NIFI-12446:
Commit c565e08ccca6c1fce17b1d3de49d7a160cfbee38 in nifi's branch
refs/heads/support/nifi-1.x from EndzeitBegins
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c565e08ccc ]
NIFI-12446 Refactor FilterAttribute to align with code conventions
This closes #8098
Signed-off-by: David Handermann
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Labels: backport-needed
> Attachments: image-2023-11-30-16-18-43-040.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12446 Refactor FilterAttribute to align with code conventions [nifi]
exceptionfactory commented on PR #8098: URL: https://github.com/apache/nifi/pull/8098#issuecomment-1839991223 Merged in https://github.com/apache/nifi/commit/c565e08ccca6c1fce17b1d3de49d7a160cfbee38 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12446 Refactor FilterAttribute to align with code conventions [nifi]
exceptionfactory closed pull request #8098: NIFI-12446 Refactor FilterAttribute to align with code conventions URL: https://github.com/apache/nifi/pull/8098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
exceptionfactory commented on code in PR #8122:
URL: https://github.com/apache/nifi/pull/8122#discussion_r1414854636
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/pom.xml:
##
@@ -26,6 +26,23 @@
jar
+
+org.apache.nifi
+nifi-api
+provided
+
+
+com.slack.api
+bolt-socket-mode
+1.32.1
Review Comment:
The latest version is 1.36.1, can this be updated?
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/pom.xml:
##
@@ -26,6 +26,23 @@
jar
+
+org.apache.nifi
+nifi-api
+provided
+
+
+com.slack.api
+bolt-socket-mode
+1.32.1
+
+
+
+org.glassfish.tyrus.bundles
+tyrus-standalone-client
+1.20
Review Comment:
The latest version of version 1 is 1.21.
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PublishSlack.java:
##
@@ -0,0 +1,489 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack;
+
+import com.slack.api.bolt.App;
+import com.slack.api.bolt.AppConfig;
+import com.slack.api.methods.MethodsClient;
+import com.slack.api.methods.SlackApiException;
+import com.slack.api.methods.request.chat.ChatPostMessageRequest;
+import com.slack.api.methods.request.files.FilesUploadV2Request;
+import com.slack.api.methods.response.chat.ChatPostMessageResponse;
+import com.slack.api.methods.response.files.FilesUploadV2Response;
+import com.slack.api.model.File;
+import com.slack.api.model.File.ShareDetail;
+import com.slack.api.model.File.Shares;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.slack.util.ChannelMapper;
+import org.apache.nifi.processors.slack.util.RateLimit;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.FormatUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.time.Duration;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Posts a message to the specified Slack channel. The
content of the message can be either a user-defined message that makes use of
Expression Language or " +
+"the contents of the FlowFile can be sent as the message. If sending a
user-defined message, the contents of the FlowFile may also be optionally
uploaded as " +
+"a file attachment.")
+@Tags({"slack", "conversation", "chat.postMessage", "social media", "team",
"text", "unstructured", "write", "upload", "send", "publish"})
+@WritesAttributes({
+@WritesAttribute(attribute = "slack.channel.id", description = "The ID of
the Slack Channel from which
Re: [PR] NIFI-11858 Configurable Column Name Normalization in PutDatabaseRecord and UpdateDatabaseTable [nifi]
ravinarayansingh commented on PR #7544: URL: https://github.com/apache/nifi/pull/7544#issuecomment-1839974560 Hi @exceptionfactory i have made the required changes please have a look Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
exceptionfactory commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414850950
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PublishSlack.java:
##
@@ -0,0 +1,517 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack;
+
+import com.slack.api.bolt.App;
+import com.slack.api.bolt.AppConfig;
+import com.slack.api.methods.MethodsClient;
+import com.slack.api.methods.request.chat.ChatPostMessageRequest;
+import com.slack.api.methods.request.files.FilesUploadV2Request;
+import com.slack.api.methods.response.chat.ChatPostMessageResponse;
+import com.slack.api.methods.response.files.FilesUploadV2Response;
+import com.slack.api.model.File;
+import com.slack.api.model.File.ShareDetail;
+import com.slack.api.model.File.Shares;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.MultiProcessorUseCase;
+import org.apache.nifi.annotation.documentation.ProcessorConfiguration;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.documentation.UseCase;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.slack.util.SlackResponseUtil;
+import org.apache.nifi.processors.slack.util.ChannelMapper;
+import org.apache.nifi.processors.slack.util.RateLimit;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.FormatUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.time.Duration;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("""
+Posts a message to the specified Slack channel. The content of the message
can be either a user-defined message that makes use of Expression Language or
+the contents of the FlowFile can be sent as the message. If sending a
user-defined message, the contents of the FlowFile may also be optionally
uploaded as
+a file attachment.
+""")
+@SeeAlso({ListenSlack.class, ConsumeSlack.class})
+@Tags({"slack", "conversation", "chat.postMessage", "social media", "team",
"text", "unstructured", "write", "upload", "send", "publish"})
+@WritesAttributes({
+@WritesAttribute(attribute = "slack.channel.id", description = "The ID of
the Slack Channel from which the messages were retrieved"),
+@WritesAttribute(attribute = "slack.ts", description = "The timestamp of
the slack messages that was sent; this is used by Slack as a unique identifier")
+})
+@DefaultSettings(yieldDuration = "3 sec")
+@UseCase(
+description = "Send specific text as a message to Slack, optionally
including the FlowFile's contents as an attached file.",
+configuration = """
+Set "Access Token" to the value of your Slack OAuth Access Token.
+Set "Channel" to the ID of the channel or the name of the channel
prefixed with the # symbol. For example, "C0123456789" or
Re: [PR] Update Dockerfile for alpine v3 / liberica v21 [nifi]
exceptionfactory commented on PR #8111: URL: https://github.com/apache/nifi/pull/8111#issuecomment-1839962723 Thanks for the thoughtful reply @hawko2600, on further review, referencing that post was short-sighted on my part. The list of vulnerabilities claim caught my eye as unsubstantiated, but I should have taken a closer look at the other claims. I also should have taken a closer at the liberica-openjdk-alpine image description, noting the glibc base instead of musl, which I expected from other Alpine containers. I see from the the [image tag details](https://hub.docker.com/layers/bellsoft/liberica-openjdk-alpine/21/images/sha256-1b160fcd879e0d7e310773aa42f3b59fa8d907c4a1a50c64c18d822541d2da93?context=explore) that this does indeed use the glibc base. This is a good example of where having a more thorough background on the change would be helped provide a better initial evaluation. If you are still willing to put time into this, I would be glad to revisit the changes. As mentioned initially, if you can create a Jira issue for tracking, that would also help capture the background rationale, and importance of the fact that this Alpine image is based on glibc as opposed to musl. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update Dockerfile for alpine v3 / liberica v21 [nifi]
joewitt commented on PR #8111: URL: https://github.com/apache/nifi/pull/8111#issuecomment-1839955747 I am still interested in this happening. If the claims arent valid we just need to run this thing. Anything which makes the builds smaller is a win but we should be consistent. We need a JIRA, and the various docker images updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update Dockerfile for alpine v3 / liberica v21 [nifi]
hawko2600 commented on PR #8111: URL: https://github.com/apache/nifi/pull/8111#issuecomment-1839944758 > @hawko2600 On further consideration, moving to Alpine does not look like a feasible approach for the standard container images. > > The primary reason is the difference in platforms with musl for Alpine versus glibc for Debian, as described in the following article: > > https://megamorf.gitlab.io/2020/05/06/why-it-s-better-not-to-use-alpine-linux-for-python-projects/ > > With NiFi 2.0.0 supporting native Python Processors, having general compatibility with Python C libraries is important. > > In terms of actual container size, the current difference between the Debian and Alpine images is around 75 MB. Although size optimization is important, NiFi binaries make up the primary share of container size. > > It is also worth highlighting that the current container images are targeted for general use cases, and particular deployment environments may have other requirements. > > There are other potential improvements for the current container configuration, so some of this discussion may be worth continuing in Jira. > > In light of the platform concerns with Alpine and Python, I am closing the pull request for now. Thanks for proposing this alternative approach. It's unfortunate that that website from 3 years ago contains verifiable lies, for example it states that there is no CVE list for alpine whereas anyone can see from the homepage of alpinelinux.org that security.alpinelinux.org is linked directly and contains a comprehensive list. I also find the performance issue contrived; the example given demonstrates installing pre-compiled apks versus compiling and installing muultiple packages from source. It claims the precompiled version took over 16 minutes whereas the needs-compiling version took only 12 seconds. There's no way that stacks up in the real world! The only way to achieve this would be to pre-compile the wheel files and include them in the docker image so the pip install only takes I/O unpacking time, which would mount to about ... 12 seconds. Obviously 16 minutes to install apks is because they set the network interface to 2Kb/s and let it download from an alpine mirror on the other side of the planet. Absolutely zero evidence was supplied as a normal shootout would do to ensure basic critical analysis would be satisfied that the premise is being tested, not external factors. The main complaint about using MUSL stdlib is that "it might" be different to glibc. Well, it better be, or they failed in their mission to remove the bloat. No evidence was produced for the claim. Presumably, if I am on a Linux system with the debug version of glibc and the debug version of musl, then the difference in debugging is that with one I run the app through gdb and the other I ... run the app through gdb? There's absolutely zero difference. It's just FUD. And in any case, it's a moot point because the images here are based on the glibc version of Alpine, which I did deliberately because I'm well aware that some libraries insist on being glibc-compatible only and we have an open-ended functionality presented through the new Python API interface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414716543
##
libminifi/src/core/extension/ExtensionManager.cpp:
##
@@ -72,9 +72,13 @@ bool ExtensionManager::initialize(const
std::shared_ptr& config) {
}));
for (const auto& candidate : candidates) {
auto library = internal::asDynamicLibrary(candidate);
- if (!library || !library->verify(logger_)) {
+ if (!library) {
continue;
}
+ if (!library->verify(logger_)) {
+logger_->log_warn("Skipping library '{}' at '{}': failed verification,
different build?",
+library->name, library->getFullPath());
+ }
Review Comment:
Yup, that was an oversight, thanks for catching it. fixed in
[bdf1cd3](https://github.com/apache/nifi-minifi-cpp/pull/1703/commits/bdf1cd3b5f27b9c4d759ebc23b1a05c259905ef9)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414719449
##
libminifi/include/core/AbstractProcessor.h:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+#include
+#include
+#include
+#include "range/v3/view/transform.hpp"
+#include "core/Annotation.h"
+#include "core/Core.h"
+#include "core/Processor.h"
+#include "core/PropertyDefinition.h"
+#include "core/RelationshipDefinition.h"
+
+namespace org::apache::nifi::minifi::core {
+template
+class AbstractProcessor : public Processor {
+ public:
+ using Processor::Processor;
+
+ void initialize() final {
+static_assert(std::is_same_v);
+static_assert(std::is_same_v);
+setSupportedProperties(ProcessorT::Properties);
+setSupportedRelationships(ProcessorT::Relationships);
+ }
+
+ void onSchedule(core::ProcessContext*, core::ProcessSessionFactory*)
override = 0;
+ void onTrigger(core::ProcessContext*, core::ProcessSession*) override = 0;
+
+ bool supportsDynamicProperties() const noexcept final { return
ProcessorT::SupportsDynamicProperties; }
+ bool supportsDynamicRelationships() const noexcept final { return
ProcessorT::SupportsDynamicRelationships; }
+ minifi::core::annotation::Input getInputRequirement() const noexcept final {
return ProcessorT::InputRequirement; }
+ bool isSingleThreaded() const noexcept final { return
ProcessorT::IsSingleThreaded; }
+ std::string getProcessorType() const final {
+constexpr auto class_name = className();
+constexpr auto last_colon_index = class_name.find_last_of(':');
+constexpr auto end = class_name.substr(last_colon_index + 1);
+if constexpr (last_colon_index == std::string_view::npos) {
+ return std::string{class_name};
+}
Review Comment:
inlined `end` to the return statement in
[bdf1cd3](https://github.com/apache/nifi-minifi-cpp/pull/1703/commits/bdf1cd3b5f27b9c4d759ebc23b1a05c259905ef9)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414719168
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](uint64_t value) { return value > 0; })
+ | utils::transform([](uint64_t value) { return size_t{value}; });
+ if (!time_window_ && !window_length_) {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Either
'Time window' or 'Window length' must be set"};
+ }
+ attribute_name_prefix_ = (context->getProperty(AttributeNamePrefix)
+ | utils::orElse([] {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION,
"'Attribute name prefix' must be set"};
+ })).value();
+ gsl_Ensures(runningInvariant());
+}
+
+void AttributeRollingWindow::onTrigger(core::ProcessContext* context,
core::ProcessSession* session) {
+ gsl_Expects(context && session && runningInvariant());
+ const auto flow_file = session->get();
+ if (!flow_file) { yield(); return; }
+ gsl_Assert(flow_file);
+ const auto current_value_opt = context->getProperty(ValueToTrack, flow_file);
+ if (!current_value_opt) {
+logger_->log_warn("Missing value to track, flow file uuid: {}",
flow_file->getUUIDStr());
+session->transfer(flow_file, Failure);
+return;
+ }
+ const auto current_value = [_value_opt] {
+try {
+ return std::stod(*current_value_opt);
+} catch (const std::exception& ex) {
+ throw minifi::Exception{ExceptionType::PROCESSOR_EXCEPTION,
+ fmt::format("Failed to convert 'Value to track' of '{}' to double",
*current_value_opt)};
+}
+ }();
+ // copy: so we can release the lock sooner
+ const auto state_copy = [&, now = std::chrono::system_clock::now()] {
+const std::lock_guard lg{state_mutex_};
+state_.add(now, current_value);
+if (window_length_) {
+ state_.shrinkToSize(*window_length_);
+} else {
+ gsl_Assert(time_window_);
+ state_.removeOlderThan(now - *time_window_);
+}
+return state_.getEntries();
+ }();
+ const auto sorted_values = [_copy] {
+auto values = state_copy |
ranges::views::transform((state_)::Entry::value) |
ranges::to;
+std::sort(std::begin(values), std::end(values));
+return values;
+ }();
+ calculateAndSetAttributes(*flow_file, sorted_values);
+ session->transfer(flow_file, Success);
+}
+
+/**
+ * Calculate statistical properties of the values in the rolling window and
set them as attributes on the flow file.
+ * Properties: count, value (sum), mean (average), median, variance, stddev
+ */
+void AttributeRollingWindow::calculateAndSetAttributes(core::FlowFile
_file,
+std::span sorted_values) const {
+ const auto attribute_name = [this](std::string_view suffix) {
+return utils::string::join_pack(attribute_name_prefix_, suffix);
+ };
+ const auto set_aggregate = [_file, attribute_name](std::string_view
name, double value) {
+flow_file.setAttribute(attribute_name(name), std::to_string(value));
+ };
+ set_aggregate("count", sorted_values.size());
+ const auto sum = std::accumulate(std::begin(sorted_values),
std::end(sorted_values), 0.0);
+ set_aggregate("value", sum);
+ const auto mean = sum / gsl::narrow_cast(sorted_values.size());
+ set_aggregate("mean", mean);
+ set_aggregate("median", [&] {
+const auto mid = sorted_values.size() / 2;
+return sorted_values.size() % 2 == 0
+? std::midpoint(sorted_values[mid], sorted_values[mid - 1]) // even
number of values: average the two
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414718950
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](uint64_t value) { return value > 0; })
+ | utils::transform([](uint64_t value) { return size_t{value}; });
+ if (!time_window_ && !window_length_) {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Either
'Time window' or 'Window length' must be set"};
+ }
+ attribute_name_prefix_ = (context->getProperty(AttributeNamePrefix)
+ | utils::orElse([] {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION,
"'Attribute name prefix' must be set"};
+ })).value();
+ gsl_Ensures(runningInvariant());
+}
+
+void AttributeRollingWindow::onTrigger(core::ProcessContext* context,
core::ProcessSession* session) {
+ gsl_Expects(context && session && runningInvariant());
+ const auto flow_file = session->get();
+ if (!flow_file) { yield(); return; }
+ gsl_Assert(flow_file);
+ const auto current_value_opt = context->getProperty(ValueToTrack, flow_file);
+ if (!current_value_opt) {
+logger_->log_warn("Missing value to track, flow file uuid: {}",
flow_file->getUUIDStr());
+session->transfer(flow_file, Failure);
+return;
+ }
+ const auto current_value = [_value_opt] {
+try {
+ return std::stod(*current_value_opt);
+} catch (const std::exception& ex) {
+ throw minifi::Exception{ExceptionType::PROCESSOR_EXCEPTION,
+ fmt::format("Failed to convert 'Value to track' of '{}' to double",
*current_value_opt)};
+}
+ }();
+ // copy: so we can release the lock sooner
+ const auto state_copy = [&, now = std::chrono::system_clock::now()] {
+const std::lock_guard lg{state_mutex_};
+state_.add(now, current_value);
+if (window_length_) {
+ state_.shrinkToSize(*window_length_);
+} else {
+ gsl_Assert(time_window_);
+ state_.removeOlderThan(now - *time_window_);
+}
+return state_.getEntries();
+ }();
+ const auto sorted_values = [_copy] {
+auto values = state_copy |
ranges::views::transform((state_)::Entry::value) |
ranges::to;
+std::sort(std::begin(values), std::end(values));
+return values;
+ }();
+ calculateAndSetAttributes(*flow_file, sorted_values);
+ session->transfer(flow_file, Success);
+}
+
+/**
+ * Calculate statistical properties of the values in the rolling window and
set them as attributes on the flow file.
+ * Properties: count, value (sum), mean (average), median, variance, stddev
Review Comment:
fixed in
[bdf1cd3](https://github.com/apache/nifi-minifi-cpp/pull/1703/commits/bdf1cd3b5f27b9c4d759ebc23b1a05c259905ef9)
(added min, max)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414718897
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](uint64_t value) { return value > 0; })
+ | utils::transform([](uint64_t value) { return size_t{value}; });
+ if (!time_window_ && !window_length_) {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Either
'Time window' or 'Window length' must be set"};
+ }
+ attribute_name_prefix_ = (context->getProperty(AttributeNamePrefix)
+ | utils::orElse([] {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION,
"'Attribute name prefix' must be set"};
+ })).value();
+ gsl_Ensures(runningInvariant());
+}
+
+void AttributeRollingWindow::onTrigger(core::ProcessContext* context,
core::ProcessSession* session) {
+ gsl_Expects(context && session && runningInvariant());
+ const auto flow_file = session->get();
+ if (!flow_file) { yield(); return; }
+ gsl_Assert(flow_file);
+ const auto current_value_opt = context->getProperty(ValueToTrack, flow_file);
+ if (!current_value_opt) {
+logger_->log_warn("Missing value to track, flow file uuid: {}",
flow_file->getUUIDStr());
+session->transfer(flow_file, Failure);
+return;
+ }
+ const auto current_value = [_value_opt] {
+try {
+ return std::stod(*current_value_opt);
+} catch (const std::exception& ex) {
+ throw minifi::Exception{ExceptionType::PROCESSOR_EXCEPTION,
+ fmt::format("Failed to convert 'Value to track' of '{}' to double",
*current_value_opt)};
+}
+ }();
Review Comment:
fixed in
[bdf1cd3](https://github.com/apache/nifi-minifi-cpp/pull/1703/commits/bdf1cd3b5f27b9c4d759ebc23b1a05c259905ef9)
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](uint64_t value) { return value > 0; })
+ | utils::transform([](uint64_t value) { return size_t{value}; });
+ if (!time_window_ && !window_length_) {
+throw
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414718702
##
extensions/standard-processors/processors/AttributeRollingWindow.h:
##
@@ -0,0 +1,117 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "core/AbstractProcessor.h"
+#include "core/Annotation.h"
+#include "core/logging/LoggerFactory.h"
+#include "core/PropertyDefinitionBuilder.h"
+#include "core/PropertyType.h"
+#include "core/RelationshipDefinition.h"
+#include "RollingWindow.h"
+#include "StateManager.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+class AttributeRollingWindow final : public
core::AbstractProcessor {
+ public:
+ using core::AbstractProcessor::AbstractProcessor;
+
+ EXTENSIONAPI static constexpr auto Description = "Track a Rolling Window
based on evaluating an Expression Language "
+ "expression on each FlowFile. Each FlowFile will be emitted with the
count of FlowFiles and total aggregate value"
+ "of values processed in the current window.";
+
+ EXTENSIONAPI static constexpr auto ValueToTrack =
core::PropertyDefinitionBuilder<>::createProperty("Value to track")
+ .withDescription("The expression on which to evaluate each FlowFile. The
result of the expression will be added "
+ "to the rolling window value.")
+ .isRequired(true)
+ .supportsExpressionLanguage(true)
+ .build();
+ EXTENSIONAPI static constexpr auto TimeWindow =
core::PropertyDefinitionBuilder<>::createProperty("Time window")
+ .withDescription("The amount of time for a rolling window. The format of
the value is expected to be a "
+ "count followed by a time unit. For example 5 millis, 10 secs, 1
min, 3 hours, 2 days, etc.")
+ .withPropertyType(core::StandardPropertyTypes::TIME_PERIOD_TYPE)
+ .build();
+ EXTENSIONAPI static constexpr auto WindowLength =
core::PropertyDefinitionBuilder<>::createProperty("Window length")
+ .withDescription("The window length in number of values. Takes
precedence over 'Time window'. If set to zero, "
+ "the 'Time window' property is used instead.")
+ .isRequired(true)
+ .withDefaultValue("0")
+ .withPropertyType(core::StandardPropertyTypes::UNSIGNED_INT_TYPE)
+ .build();
+ EXTENSIONAPI static constexpr auto AttributeNamePrefix =
core::PropertyDefinitionBuilder<>::createProperty("Attribute name prefix")
+ .withDescription("The prefix to add to the generated attribute names.
For example, if this is set to 'rolling.window.', "
+ "then the full attribute names will be
'rolling.window.value', 'rolling.window.count', etc.")
+ .isRequired(true)
+ .withDefaultValue("rolling.window.")
+ .build();
+ EXTENSIONAPI static constexpr auto Properties =
std::array{
+ValueToTrack,
+TimeWindow,
+WindowLength,
+AttributeNamePrefix
+ };
+
+ EXTENSIONAPI static constexpr auto Success =
core::RelationshipDefinition{"success", "All FlowFiles that are "
+ "successfully processed are routed to this relationship."};
+ EXTENSIONAPI static constexpr auto Failure =
core::RelationshipDefinition{"failure", "When a FlowFile fails, "
+ "it is routed here."};
+ EXTENSIONAPI static constexpr auto Relationships = std::array{Success,
Failure};
+
+ EXTENSIONAPI static constexpr auto Count =
core::OutputAttributeDefinition<1>{"count", {Success}, "Number of the
values in the rolling window"};
+ EXTENSIONAPI static constexpr auto Value =
core::OutputAttributeDefinition<1>{"value", {Success}, "Sum of the
values in the rolling window"};
+ EXTENSIONAPI static constexpr auto Mean =
core::OutputAttributeDefinition<1>{"mean", {Success}, "Mean of the
values in the rolling window"};
+ EXTENSIONAPI static constexpr auto Median =
core::OutputAttributeDefinition<1>{"median", {Success}, "Median of the
values in the rolling window"};
+ EXTENSIONAPI static constexpr auto Variance =
core::OutputAttributeDefinition<1>{"variance", {Success}, "Variance of
the values in the rolling window"};
+ EXTENSIONAPI static constexpr auto Stddev =
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703: URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414718561 ## PROCESSORS.md: ## @@ -147,6 +148,44 @@ In the list below, the names of required properties appear in bold. Any other pr | success | success operational on the flow record | +## AttributeRollingWindow + +### Description + +Track a Rolling Window based on evaluating an Expression Language expression on each FlowFile. Each FlowFile will be emitted with the count of FlowFiles and total aggregate valueof values processed in the current window. Review Comment: fixed in [bdf1cd3](https://github.com/apache/nifi-minifi-cpp/pull/1703/commits/bdf1cd3b5f27b9c4d759ebc23b1a05c259905ef9) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414716543
##
libminifi/src/core/extension/ExtensionManager.cpp:
##
@@ -72,9 +72,13 @@ bool ExtensionManager::initialize(const
std::shared_ptr& config) {
}));
for (const auto& candidate : candidates) {
auto library = internal::asDynamicLibrary(candidate);
- if (!library || !library->verify(logger_)) {
+ if (!library) {
continue;
}
+ if (!library->verify(logger_)) {
+logger_->log_warn("Skipping library '{}' at '{}': failed verification,
different build?",
+library->name, library->getFullPath());
+ }
Review Comment:
Yup, that was an oversight, thanks for catching it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703: URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414711038 ## PROCESSORS.md: ## @@ -147,6 +148,44 @@ In the list below, the names of required properties appear in bold. Any other pr | success | success operational on the flow record | +## AttributeRollingWindow + +### Description + +Track a Rolling Window based on evaluating an Expression Language expression on each FlowFile. Each FlowFile will be emitted with the count of FlowFiles and total aggregate valueof values processed in the current window. + +### Properties + +In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language. + +| Name | Default Value | Allowable Values | Description | +|---|-|--|--| +| **Value to track**| | | The expression on which to evaluate each FlowFile. The result of the expression will be added to the rolling window value.**Supports Expression Language: true**| +| Time window | | | The amount of time for a rolling window. The format of the value is expected to be a count followed by a time unit. For example 5 millis, 10 secs, 1 min, 3 hours, 2 days, etc. | +| **Window length** | 0 | | The window length in number of values. Takes precedence over 'Time window'. If set to zero, the 'Time window' property is used instead. | +| **Attribute name prefix** | rolling.window. | | The prefix to add to the generated attribute names. For example, if this is set to 'rolling.window.', then the full attribute names will be 'rolling.window.value', 'rolling.window.count', etc. | + +### Relationships + +| Name| Description | +|-|| +| success | All FlowFiles that are successfully processed are routed to this relationship. | +| failure | When a FlowFile fails, it is routed here. | + +### Output Attributes + +| Attribute| Relationship | Description | +|--|--|| +| count| success | Number of the values in the rolling window | +| value| success | Sum of the values in the rolling window | Review Comment: The processor already quite different from the NiFi version, but I prefer not to deviate in this aspect. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414710024
##
libminifi/include/core/AbstractProcessor.h:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+#include
+#include
+#include
+#include "range/v3/view/transform.hpp"
+#include "core/Annotation.h"
+#include "core/Core.h"
+#include "core/Processor.h"
+#include "core/PropertyDefinition.h"
+#include "core/RelationshipDefinition.h"
+
+namespace org::apache::nifi::minifi::core {
+template
+class AbstractProcessor : public Processor {
+ public:
+ using Processor::Processor;
+
+ void initialize() final {
+static_assert(std::is_same_v);
+static_assert(std::is_same_v);
+setSupportedProperties(ProcessorT::Properties);
+setSupportedRelationships(ProcessorT::Relationships);
+ }
+
+ void onSchedule(core::ProcessContext*, core::ProcessSessionFactory*)
override = 0;
+ void onTrigger(core::ProcessContext*, core::ProcessSession*) override = 0;
+
+ bool supportsDynamicProperties() const noexcept final { return
ProcessorT::SupportsDynamicProperties; }
+ bool supportsDynamicRelationships() const noexcept final { return
ProcessorT::SupportsDynamicRelationships; }
+ minifi::core::annotation::Input getInputRequirement() const noexcept final {
return ProcessorT::InputRequirement; }
+ bool isSingleThreaded() const noexcept final { return
ProcessorT::IsSingleThreaded; }
+ std::string getProcessorType() const final {
+constexpr auto class_name = className();
+constexpr auto last_colon_index = class_name.find_last_of(':');
+constexpr auto end = class_name.substr(last_colon_index + 1);
+if constexpr (last_colon_index == std::string_view::npos) {
+ return std::string{class_name};
+}
Review Comment:
Why would it not compile? I have nothing against switching it, but I think
all compliant compilers should have constexpr string_view::substr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414704269
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](uint64_t value) { return value > 0; })
+ | utils::transform([](uint64_t value) { return size_t{value}; });
Review Comment:
It's using direct-list-initialization, so narrowing conversions will fail to
compile. I originally didn't consider 32 bit platforms, but it would probably
be better to not break the build there. I'm changing it to gsl::narrow, so a
narrowing conversion will terminate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414703188
##
extensions/standard-processors/RollingWindow.h:
##
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+
+namespace org::apache::nifi::minifi::processors::standard::utils {
+
+namespace detail {
+template
+struct priority_queue : std::priority_queue {
+ using std::priority_queue::priority_queue;
+
+ // Expose the underlying container
+ const Container& get_container() const & { return this->c; }
+ Container get_container() && { return std::move(this->c); }
+};
+} // namespace detail
+
+template
+class RollingWindow {
+ public:
+ struct Entry {
+Timestamp timestamp{};
+Value value{};
+ };
+ struct EntryComparator {
+// greater-than, because std::priority_queue order is reversed. This way,
top() is the oldest entry.
+bool operator()(const Entry& lhs, const Entry& rhs) const {
+ return lhs.timestamp > rhs.timestamp;
+}
+ };
+
+ void removeOlderThan(Timestamp timestamp) {
+while (!state_.empty() && state_.top().timestamp < timestamp) {
+ state_.pop();
+}
+ }
+
+ /** Remove the oldest entries until the size is <= size. */
+ void shrinkToSize(size_t size) {
+while (state_.size() > size && !state_.empty()) {
Review Comment:
While the AttributeRollingWindow processor never calls this with a size of
0, the interface contract of RollingWindow allows shrinking to 0. In the case
of AttributeRollingWindow, the optimizer can probably prove that size is never
zero, and get rid of the extra check after inlining.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414699559
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,121 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](size_t value) { return value > 0; });
+ if (!time_window_ && !window_length_) {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Either
'Time window' or 'Window length' must be set"};
+ }
+ attribute_name_prefix_ = (context->getProperty(AttributeNamePrefix)
+ | utils::orElse([] {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION,
"'Attribute name prefix' must be set"};
+ })).value();
+ gsl_Ensures(runningInvariant());
+}
+
+void AttributeRollingWindow::onTrigger(core::ProcessContext* context,
core::ProcessSession* session) {
+ gsl_Expects(context && session && runningInvariant());
+ const auto flow_file = session->get();
+ if (!flow_file) { yield(); return; }
+ gsl_Assert(flow_file);
+ const auto current_value_opt = context->getProperty(ValueToTrack, flow_file);
+ if (!current_value_opt) {
+logger_->log_warn("Missing value to track, flow file uuid: {}",
flow_file->getUUIDStr());
+session->transfer(flow_file, Failure);
+return;
+ }
+ const auto current_value = [_value_opt] {
+try {
+ return std::stod(*current_value_opt);
+} catch (const std::exception& ex) {
+ throw minifi::Exception{ExceptionType::PROCESSOR_EXCEPTION,
+ fmt::format("Failed to convert 'Value to track' of '{}' to double",
*current_value_opt)};
+}
+ }();
+ // copy: so we can release the lock sooner
+ const auto state_copy = [&, now = std::chrono::system_clock::now()] {
+const std::lock_guard lg{state_mutex_};
+state_.add(now, current_value);
+if (window_length_) {
+ state_.shrinkToSize(*window_length_);
+} else {
+ gsl_Assert(time_window_);
+ state_.removeOlderThan(now - *time_window_);
+}
+return state_.getEntries();
+ }();
+ const auto sorted_values = [_copy] {
+auto values = state_copy |
ranges::views::transform((state_)::Entry::value) |
ranges::to;
+std::sort(std::begin(values), std::end(values));
+return values;
+ }();
+ calculateAndSetAttributes(*flow_file, sorted_values);
+ session->transfer(flow_file, Success);
+}
+
+/**
+ * Calculate statistical properties of the values in the rolling window and
set them as attributes on the flow file.
+ * Properties: count, value (sum), mean (average), median, variance, stddev
+ */
+void AttributeRollingWindow::calculateAndSetAttributes(core::FlowFile
_file,
+std::span sorted_values) const {
+ const auto attribute_name = [this](std::string_view suffix) {
+return utils::string::join_pack(attribute_name_prefix_, suffix);
+ };
+ const auto set_aggregate = [_file, attribute_name](std::string_view
name, double value) {
+flow_file.setAttribute(attribute_name(name), std::to_string(value));
+ };
+ set_aggregate("count", sorted_values.size());
+ const auto sum = std::accumulate(std::begin(sorted_values),
std::end(sorted_values), 0.0);
+ set_aggregate("value", sum);
+ const auto mean = sum / gsl::narrow_cast(sorted_values.size());
+ set_aggregate("mean", mean);
+ set_aggregate("median", [&] {
+const auto mid = sorted_values.size() / 2;
+return sorted_values.size() % 2 == 0
+? std::midpoint(sorted_values[mid], sorted_values[mid - 1]) // even
number of values: average the two middle values
+: sorted_values[mid]; // odd number of values:
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
szaszm commented on code in PR #1703:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1703#discussion_r1414698106
##
extensions/standard-processors/processors/AttributeRollingWindow.cpp:
##
@@ -0,0 +1,121 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "AttributeRollingWindow.h"
+#include
+#include
+#include "fmt/format.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/expected.h"
+#include "utils/OptionalUtils.h"
+
+namespace org::apache::nifi::minifi::processors {
+
+void AttributeRollingWindow::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory*) {
+ gsl_Expects(context);
+ time_window_ = context->getProperty(TimeWindow)
+ | utils::transform(::TimePeriodValue::getMilliseconds);
+ window_length_ = context->getProperty(WindowLength)
+ | utils::filter([](size_t value) { return value > 0; });
+ if (!time_window_ && !window_length_) {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Either
'Time window' or 'Window length' must be set"};
+ }
+ attribute_name_prefix_ = (context->getProperty(AttributeNamePrefix)
+ | utils::orElse([] {
+throw minifi::Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION,
"'Attribute name prefix' must be set"};
+ })).value();
+ gsl_Ensures(runningInvariant());
+}
+
+void AttributeRollingWindow::onTrigger(core::ProcessContext* context,
core::ProcessSession* session) {
+ gsl_Expects(context && session && runningInvariant());
+ const auto flow_file = session->get();
+ if (!flow_file) { yield(); return; }
+ gsl_Assert(flow_file);
+ const auto current_value_opt = context->getProperty(ValueToTrack, flow_file);
+ if (!current_value_opt) {
+logger_->log_warn("Missing value to track, flow file uuid: {}",
flow_file->getUUIDStr());
+session->transfer(flow_file, Failure);
+return;
+ }
+ const auto current_value = [_value_opt] {
+try {
+ return std::stod(*current_value_opt);
+} catch (const std::exception& ex) {
+ throw minifi::Exception{ExceptionType::PROCESSOR_EXCEPTION,
+ fmt::format("Failed to convert 'Value to track' of '{}' to double",
*current_value_opt)};
+}
+ }();
+ // copy: so we can release the lock sooner
+ const auto state_copy = [&, now = std::chrono::system_clock::now()] {
+const std::lock_guard lg{state_mutex_};
+state_.add(now, current_value);
+if (window_length_) {
+ state_.shrinkToSize(*window_length_);
+} else {
+ gsl_Assert(time_window_);
+ state_.removeOlderThan(now - *time_window_);
+}
+return state_.getEntries();
+ }();
+ const auto sorted_values = [_copy] {
+auto values = state_copy |
ranges::views::transform((state_)::Entry::value) |
ranges::to;
+std::sort(std::begin(values), std::end(values));
+return values;
+ }();
+ calculateAndSetAttributes(*flow_file, sorted_values);
+ session->transfer(flow_file, Success);
+}
+
+/**
+ * Calculate statistical properties of the values in the rolling window and
set them as attributes on the flow file.
+ * Properties: count, value (sum), mean (average), median, variance, stddev
+ */
+void AttributeRollingWindow::calculateAndSetAttributes(core::FlowFile
_file,
+std::span sorted_values) const {
+ const auto attribute_name = [this](std::string_view suffix) {
+return utils::string::join_pack(attribute_name_prefix_, suffix);
+ };
+ const auto set_aggregate = [_file, attribute_name](std::string_view
name, double value) {
+flow_file.setAttribute(attribute_name(name), std::to_string(value));
+ };
+ set_aggregate("count", sorted_values.size());
+ const auto sum = std::accumulate(std::begin(sorted_values),
std::end(sorted_values), 0.0);
Review Comment:
I prefer the standard library versions when available, because the headers
are more lightweight, and it's not much more typing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure
[jira] [Resolved] (NIFI-12461) Update jsoup to 1.17.1
[ https://issues.apache.org/jira/browse/NIFI-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12461. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update jsoup to 1.17.1 > -- > > Key: NIFI-12461 > URL: https://issues.apache.org/jira/browse/NIFI-12461 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update jsoup to 1.17.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12463) Update maven-artifact to 3.8.8
[ https://issues.apache.org/jira/browse/NIFI-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12463. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update maven-artifact to 3.8.8 > -- > > Key: NIFI-12463 > URL: https://issues.apache.org/jira/browse/NIFI-12463 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update maven-artifact to 3.8.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12465) Update mysql-binlog-connector-java to 0.28.3
[ https://issues.apache.org/jira/browse/NIFI-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12465: Priority: Minor (was: Major) > Update mysql-binlog-connector-java to 0.28.3 > > > Key: NIFI-12465 > URL: https://issues.apache.org/jira/browse/NIFI-12465 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.28.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12465) Update mysql-binlog-connector-java to 0.28.3
[ https://issues.apache.org/jira/browse/NIFI-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12465. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update mysql-binlog-connector-java to 0.28.3 > > > Key: NIFI-12465 > URL: https://issues.apache.org/jira/browse/NIFI-12465 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.28.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12466) Update mongo.driver.version to 4.11.1
[ https://issues.apache.org/jira/browse/NIFI-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12466: Priority: Minor (was: Major) > Update mongo.driver.version to 4.11.1 > - > > Key: NIFI-12466 > URL: https://issues.apache.org/jira/browse/NIFI-12466 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update mongo.driver.version to 4.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12461) Update jsoup to 1.17.1
[ https://issues.apache.org/jira/browse/NIFI-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12461: Priority: Minor (was: Major) > Update jsoup to 1.17.1 > -- > > Key: NIFI-12461 > URL: https://issues.apache.org/jira/browse/NIFI-12461 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update jsoup to 1.17.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12466) Update mongo.driver.version to 4.11.1
[ https://issues.apache.org/jira/browse/NIFI-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12466. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update mongo.driver.version to 4.11.1 > - > > Key: NIFI-12466 > URL: https://issues.apache.org/jira/browse/NIFI-12466 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update mongo.driver.version to 4.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12460) Update OpenCSV to 5.9
[ https://issues.apache.org/jira/browse/NIFI-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12460. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update OpenCSV to 5.9 > - > > Key: NIFI-12460 > URL: https://issues.apache.org/jira/browse/NIFI-12460 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update OpenCSV to 5.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12463) Update maven-artifact to 3.8.8
[ https://issues.apache.org/jira/browse/NIFI-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793059#comment-17793059 ] ASF subversion and git services commented on NIFI-12463: Commit 0101bf8f176d0d2f5ca29bd559952549a657e491 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0101bf8f17 ] NIFI-12463 Upgraded maven-artifact from 3.8.5 to 3.8.8 This closes #8115 Signed-off-by: David Handermann (cherry picked from commit 29dda4208a47c15476363f09911a4cc6bf4cd362) > Update maven-artifact to 3.8.8 > -- > > Key: NIFI-12463 > URL: https://issues.apache.org/jira/browse/NIFI-12463 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update maven-artifact to 3.8.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12465) Update mysql-binlog-connector-java to 0.28.3
[ https://issues.apache.org/jira/browse/NIFI-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793060#comment-17793060 ] ASF subversion and git services commented on NIFI-12465: Commit 9d1a8c5556b377584ee2d735b3be567b6ed79075 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9d1a8c5556 ] NIFI-12465 Upgraded mysql-binlog-connector-java from 0.28.1 to 0.28.3 This closes #8117 Signed-off-by: David Handermann (cherry picked from commit d6bac808ba8aa13272eab6df00a0e332a74175aa) > Update mysql-binlog-connector-java to 0.28.3 > > > Key: NIFI-12465 > URL: https://issues.apache.org/jira/browse/NIFI-12465 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.28.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12466) Update mongo.driver.version to 4.11.1
[ https://issues.apache.org/jira/browse/NIFI-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793061#comment-17793061 ] ASF subversion and git services commented on NIFI-12466: Commit b13e41ebbad9f739cf98cf2876f29b5b91f0be54 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b13e41ebba ] NIFI-12466 Upgraded MongoDB Driver from 4.10.2 to 4.11.1 This closes #8118 Signed-off-by: David Handermann (cherry picked from commit d1c7e6c6b8d176d026a33f15eca277ef8c20b81a) > Update mongo.driver.version to 4.11.1 > - > > Key: NIFI-12466 > URL: https://issues.apache.org/jira/browse/NIFI-12466 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update mongo.driver.version to 4.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12460) Update OpenCSV to 5.9
[ https://issues.apache.org/jira/browse/NIFI-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793057#comment-17793057 ] ASF subversion and git services commented on NIFI-12460: Commit edafb452bcf9c2a5529a89c0dfa3f8401888d3b8 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=edafb452bc ] NIFI-12460 Upgraded OpenCSV from 5.8 to 5.9 This closes #8112 Signed-off-by: David Handermann (cherry picked from commit 53f932bdecdfb2e0c819dbd603a0ef53e94b84fa) > Update OpenCSV to 5.9 > - > > Key: NIFI-12460 > URL: https://issues.apache.org/jira/browse/NIFI-12460 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update OpenCSV to 5.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12455) Update jaxb.runtime.version to 2.3.9
[ https://issues.apache.org/jira/browse/NIFI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793056#comment-17793056 ] ASF subversion and git services commented on NIFI-12455: Commit a1cf0787186a2cd8a4fcd8672342991bb72e649a in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a1cf078718 ] NIFI-12455 Upgraded JAXB Runtime from 2.3.8 to 2.3.9 This closes #8106 Signed-off-by: David Handermann (cherry picked from commit 30c139000ceeb495c82e04be011ff8c197852977) > Update jaxb.runtime.version to 2.3.9 > > > Key: NIFI-12455 > URL: https://issues.apache.org/jira/browse/NIFI-12455 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.24.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jaxb.runtime.version to 2.3.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12461) Update jsoup to 1.17.1
[ https://issues.apache.org/jira/browse/NIFI-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793058#comment-17793058 ] ASF subversion and git services commented on NIFI-12461: Commit e5ce57d7637f6e89be62d86bd254d0b257f27856 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e5ce57d763 ] NIFI-12461 Upgraded jsoup from 1.16.2 to 1.17.1 This closes #8113 Signed-off-by: David Handermann (cherry picked from commit d180ff936693a3fbdd3fdce025d771eea3640cb0) > Update jsoup to 1.17.1 > -- > > Key: NIFI-12461 > URL: https://issues.apache.org/jira/browse/NIFI-12461 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jsoup to 1.17.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12455) Update jaxb.runtime.version to 2.3.9
[ https://issues.apache.org/jira/browse/NIFI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12455. - Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Update jaxb.runtime.version to 2.3.9 > > > Key: NIFI-12455 > URL: https://issues.apache.org/jira/browse/NIFI-12455 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.24.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update jaxb.runtime.version to 2.3.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12463) Update maven-artifact to 3.8.8
[ https://issues.apache.org/jira/browse/NIFI-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793053#comment-17793053 ] ASF subversion and git services commented on NIFI-12463: Commit 29dda4208a47c15476363f09911a4cc6bf4cd362 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=29dda4208a ] NIFI-12463 Upgraded maven-artifact from 3.8.5 to 3.8.8 This closes #8115 Signed-off-by: David Handermann > Update maven-artifact to 3.8.8 > -- > > Key: NIFI-12463 > URL: https://issues.apache.org/jira/browse/NIFI-12463 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update maven-artifact to 3.8.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12461) Update jsoup to 1.17.1
[ https://issues.apache.org/jira/browse/NIFI-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793052#comment-17793052 ] ASF subversion and git services commented on NIFI-12461: Commit d180ff936693a3fbdd3fdce025d771eea3640cb0 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d180ff9366 ] NIFI-12461 Upgraded jsoup from 1.16.2 to 1.17.1 This closes #8113 Signed-off-by: David Handermann > Update jsoup to 1.17.1 > -- > > Key: NIFI-12461 > URL: https://issues.apache.org/jira/browse/NIFI-12461 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jsoup to 1.17.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12455) Update jaxb.runtime.version to 2.3.9
[ https://issues.apache.org/jira/browse/NIFI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793050#comment-17793050 ] ASF subversion and git services commented on NIFI-12455: Commit 30c139000ceeb495c82e04be011ff8c197852977 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=30c139000c ] NIFI-12455 Upgraded JAXB Runtime from 2.3.8 to 2.3.9 This closes #8106 Signed-off-by: David Handermann > Update jaxb.runtime.version to 2.3.9 > > > Key: NIFI-12455 > URL: https://issues.apache.org/jira/browse/NIFI-12455 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.24.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jaxb.runtime.version to 2.3.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12465) Update mysql-binlog-connector-java to 0.28.3
[ https://issues.apache.org/jira/browse/NIFI-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793054#comment-17793054 ] ASF subversion and git services commented on NIFI-12465: Commit d6bac808ba8aa13272eab6df00a0e332a74175aa in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d6bac808ba ] NIFI-12465 Upgraded mysql-binlog-connector-java from 0.28.1 to 0.28.3 This closes #8117 Signed-off-by: David Handermann > Update mysql-binlog-connector-java to 0.28.3 > > > Key: NIFI-12465 > URL: https://issues.apache.org/jira/browse/NIFI-12465 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.28.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12460) Update OpenCSV to 5.9
[ https://issues.apache.org/jira/browse/NIFI-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793051#comment-17793051 ] ASF subversion and git services commented on NIFI-12460: Commit 53f932bdecdfb2e0c819dbd603a0ef53e94b84fa in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=53f932bdec ] NIFI-12460 Upgraded OpenCSV from 5.8 to 5.9 This closes #8112 Signed-off-by: David Handermann > Update OpenCSV to 5.9 > - > > Key: NIFI-12460 > URL: https://issues.apache.org/jira/browse/NIFI-12460 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update OpenCSV to 5.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12466) Update mongo.driver.version to 4.11.1
[ https://issues.apache.org/jira/browse/NIFI-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793055#comment-17793055 ] ASF subversion and git services commented on NIFI-12466: Commit d1c7e6c6b8d176d026a33f15eca277ef8c20b81a in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d1c7e6c6b8 ] NIFI-12466 Upgraded MongoDB Driver from 4.10.2 to 4.11.1 This closes #8118 Signed-off-by: David Handermann > Update mongo.driver.version to 4.11.1 > - > > Key: NIFI-12466 > URL: https://issues.apache.org/jira/browse/NIFI-12466 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update mongo.driver.version to 4.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12461 Update jsoup to 1.17.1 [nifi]
exceptionfactory closed pull request #8113: NIFI-12461 Update jsoup to 1.17.1 URL: https://github.com/apache/nifi/pull/8113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12466 Update mongo.driver.version to 4.11.1 [nifi]
exceptionfactory closed pull request #8118: NIFI-12466 Update mongo.driver.version to 4.11.1 URL: https://github.com/apache/nifi/pull/8118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12463 Update maven-artifact to 3.8.8 [nifi]
exceptionfactory closed pull request #8115: NIFI-12463 Update maven-artifact to 3.8.8 URL: https://github.com/apache/nifi/pull/8115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12465 Update mysql-binlog-connector-java to 0.28.3 [nifi]
exceptionfactory closed pull request #8117: NIFI-12465 Update mysql-binlog-connector-java to 0.28.3 URL: https://github.com/apache/nifi/pull/8117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12460 Update OpenCSV to 5.9 [nifi]
exceptionfactory closed pull request #8112: NIFI-12460 Update OpenCSV to 5.9 URL: https://github.com/apache/nifi/pull/8112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12455 Update jaxb.runtime.version to 2.3.9 [nifi]
exceptionfactory closed pull request #8106: NIFI-12455 Update jaxb.runtime.version to 2.3.9 URL: https://github.com/apache/nifi/pull/8106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12464 Update commons-compiler to 3.1.11 [nifi]
exceptionfactory commented on PR #8116: URL: https://github.com/apache/nifi/pull/8116#issuecomment-1839687771 The lack of upgrading janino having the same version indicates why this pull request failed to build on all platforms. I am closing this for now, but if you can evaluate upgrading those components together and confirm a successful build, this should be ready to submit again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12464 Update commons-compiler to 3.1.11 [nifi]
exceptionfactory closed pull request #8116: NIFI-12464 Update commons-compiler to 3.1.11 URL: https://github.com/apache/nifi/pull/8116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12331) Introduce a PublishSlack processor
[ https://issues.apache.org/jira/browse/NIFI-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-12331: - Assignee: Mark Payne > Introduce a PublishSlack processor > -- > > Key: NIFI-12331 > URL: https://issues.apache.org/jira/browse/NIFI-12331 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > The Slack API provides multiple different ways to publish messages to a Slack > channel. NiFi already has two Processors for pushing to Slack - PostSlack and > PutSlack. These processors have slightly different nuances, and the > documentation does not articulate when to which one. One of them is oriented > more toward sending FlowFile contents as an attachment while the other is > oriented toward posting a message based on a property value. We should > consolidate both of these Processors into a single Processor that is capable > of sending a message and optionally providing the FlowFile content as an > attachment. > Both PostSlack and PutSlack make use of WebHooks instead of using the > official Slack SDK. This means that rather than simply specifying the name of > the Channel to post to, in order to send a message in Slack, the creator of > the Slack App must explicitly add a Webhook for the desired channel, and the > Processor must then be configured to use that Webhook. As a result, the > channel cannot be easily configured and cannot be dynamic. This makes it > difficult to use in conjunction with ListenSlack / ConsumeSlack in order to > respond in threads. > We need to consolidate both into a single Processor that is configured and > behaves differently, based on the SDK. > This Processor should be configured with properties that allow specifying: > * Bot Token > * Name of the channel to send to > * How to obtain the message content (FlowFile Content or specified as a > Property that accepts Expression Language) > * If using a Property value, should be configured also with the message to > send, and whether or not to attach the FlowFile content as an attachment to > the message. > * Thread Timestamp (optional to convey which thread the message should be > sent to) - should support Expression Language -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12469) Cannot configure registry client (Core Nifi)
[ https://issues.apache.org/jira/browse/NIFI-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793046#comment-17793046 ] David Handermann commented on NIFI-12469: - Thanks for the reply [~wpkinzel], that is good to know a workaround is an option. So it sounds like the problem is related to the order of creating the components? That seems likely given that Registry Clients do not have the same type of lifecycle as other components. > Cannot configure registry client (Core Nifi) > > > Key: NIFI-12469 > URL: https://issues.apache.org/jira/browse/NIFI-12469 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1 >Reporter: Bill Kinzel >Priority: Major > Attachments: nifi.JPG, stacktrace.txt > > > Nav = Nifi Settings > Registry Clients (Add) > There is an issue with configuring the SSL Context Service. After adding a > new service, the UI displays the ID, not the name. The service is created > but the client config remains in a warning state. > Message states > 'SSL Context Service' validated against > '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references > a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 commented on PR #8120: URL: https://github.com/apache/nifi/pull/8120#issuecomment-1839649244 That's a good call @exceptionfactory! Just pushed a new PR for the 1.x line: https://github.com/apache/nifi/pull/8122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 opened a new pull request, #8122: URL: https://github.com/apache/nifi/pull/8122 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12451) Remove Bootstrap Notification Services
[ https://issues.apache.org/jira/browse/NIFI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793041#comment-17793041 ] ASF subversion and git services commented on NIFI-12451: Commit 456cf7d88daf4faf367343e639e7df2b965de213 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=456cf7d88d ] NIFI-12451 This closes #8104. Removed Bootstrap Notification Services - Refactored Security Property handling and bootstrap certificate generation - Removed unnecessary dependencies from nifi-bootstrap - Removed unnecessary and unused settings from bootstrap.conf Signed-off-by: Joseph Witt > Remove Bootstrap Notification Services > -- > > Key: NIFI-12451 > URL: https://issues.apache.org/jira/browse/NIFI-12451 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 2.0.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Following the deprecation of email and HTTP notification services on the > version 1 support branch, the Notification Service components should be > removed, and configuration elements should be removed from the bootstrap.conf > configuration. Additional bootstrap.conf settings for optional components > should also be reviewed and removed to streamline the default bootstrap.conf > settings. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12451) Remove Bootstrap Notification Services
[ https://issues.apache.org/jira/browse/NIFI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12451: Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Remove Bootstrap Notification Services > -- > > Key: NIFI-12451 > URL: https://issues.apache.org/jira/browse/NIFI-12451 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 2.0.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Following the deprecation of email and HTTP notification services on the > version 1 support branch, the Notification Service components should be > removed, and configuration elements should be removed from the bootstrap.conf > configuration. Additional bootstrap.conf settings for optional components > should also be reviewed and removed to streamline the default bootstrap.conf > settings. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
asfgit closed pull request #8104: NIFI-12451 Remove Bootstrap Notification Services URL: https://github.com/apache/nifi/pull/8104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12470: Fixing forEach callback for usage with Object.entries() to address layout issue in Status History [nifi]
mcgilman opened a new pull request, #8121: URL: https://github.com/apache/nifi/pull/8121 NIFI-12470: - Fixing forEach callback for usage with Object.entries() to address layout issue in Status History. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
joewitt commented on code in PR #8120: URL: https://github.com/apache/nifi/pull/8120#discussion_r1414551223 ## nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PutSlack.java: ## @@ -66,6 +67,13 @@ @DynamicProperty(name = "A JSON object to add to Slack's \"attachments\" JSON payload.", value = "JSON-formatted string to add to Slack's payload JSON appended to the \"attachments\" JSON array.", expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Converts the contents of each value specified by the Dynamic Property's value to JSON and appends it to the payload being sent to Slack.") +@DeprecationNotice( Review Comment: Agreed. There shouldn't be anything deprecated on 2.x for a while. But another PR should deprecate these for the support branch. And this PR should remove them. Then the migration guide/component mapping guide for 2.x updated. This is cool though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12470) UI - Fix layout/formatting in Status History
[ https://issues.apache.org/jira/browse/NIFI-12470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12470: --- Attachment: Screenshot 2023-12-04 at 4.48.29 PM.png > UI - Fix layout/formatting in Status History > > > Key: NIFI-12470 > URL: https://issues.apache.org/jira/browse/NIFI-12470 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Attachments: Screenshot 2023-12-04 at 4.48.29 PM.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12470) UI - Fix layout/formatting in Status History
Matt Gilman created NIFI-12470: -- Summary: UI - Fix layout/formatting in Status History Key: NIFI-12470 URL: https://issues.apache.org/jira/browse/NIFI-12470 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Attachments: Screenshot 2023-12-04 at 4.48.29 PM.png -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
exceptionfactory commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414545297
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PostSlack.java:
##
@@ -82,6 +83,14 @@
" The property name will not be used by the processor.",
expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
@WritesAttribute(attribute="slack.file.url", description = "The Slack URL of
the uploaded file. It will be added if 'Upload FlowFile' has been set to
'Yes'.")
+@DeprecationNotice(
+alternatives = {PublishSlack.class},
+reason = "This Processor is more difficult to configure than necessary,
and exposes intricate nuances of the Slack API that need not be exposed. " +
+ "Additionally, it lacks some capabilities, such as responding to
messages in threads. It interacts directly with the Slack API instead of " +
+ "making use of the Slack-provided SDK, which makes the Processor
much more brittle. Additionally, it does not make clear the differences " +
+ "between PutSlack and PostSlack. Each provides slightly different
capabilities. " +
+ "PublishSlack combines the capabilities of both Processors in a
simpler configuration that makes use of Slack's standard REST API."
+)
Review Comment:
Rather than tag this on deprecated on the main branch, the deprecation
should be added on the support branch. The removal of these Processors could be
handled in this pull request, or in a subsequent PR.
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/pom.xml:
##
@@ -61,10 +61,6 @@
org.slf4j
jcl-over-slf4j
-
-org.apache.nifi
-nifi-api
-
Review Comment:
It seems like this should still be declared, perhaps the removal triggered
the other addition with the compile scope, but this one looks correct.
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PutSlack.java:
##
@@ -66,6 +67,13 @@
@DynamicProperty(name = "A JSON object to add to Slack's \"attachments\" JSON
payload.", value = "JSON-formatted string to add to Slack's payload JSON
appended to the \"attachments\" JSON array.",
expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES,
description = "Converts the contents of each value specified by the
Dynamic Property's value to JSON and appends it to the payload being sent to
Slack.")
+@DeprecationNotice(
Review Comment:
Same as noted on `PostSlack`, the deprecation should be removed and applied
to the support branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414512781
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
+return;
+}
+} else {
+// The character didn't match the expected one in the record
separator, reset the separator matcher
+// and check if it is the first character of the separator.
+indexIntoSeparator = 0;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+// This character is the beginning of the record
separator, keep it
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string
built so far
+return;
+}
+}
+}
Review Comment:
Instead of all the comments regarding reaching the end of a record, make a
method which conveys that intention.
```
char nextChar = (char)code;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength) ) {
return;
}
} else {
// The character didn't match the expected one in the record separator,
reset the separator matcher
// and check if it is the first character of the separator.
indexIntoSeparator = 0;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength)
) {
return;
}
}
}
private boolean hasReachedEndOfRecord(int indexIntoSeparator, int
recordSeparatorLength) {
return indexIntoSeparator == recordSeparatorLength;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12469) Cannot configure registry client (Core Nifi)
[ https://issues.apache.org/jira/browse/NIFI-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793039#comment-17793039 ] Bill Kinzel commented on NIFI-12469: Hi David, It is a new registry setup, service and all. I did find a workaround. You'll need to first create the management service, then select it in the drop-menu while setting up the registry client. I selected 'Create a new service' while setting up the client. Anyway, everything seems to working great now. > Cannot configure registry client (Core Nifi) > > > Key: NIFI-12469 > URL: https://issues.apache.org/jira/browse/NIFI-12469 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1 >Reporter: Bill Kinzel >Priority: Major > Attachments: nifi.JPG, stacktrace.txt > > > Nav = Nifi Settings > Registry Clients (Add) > There is an issue with configuring the SSL Context Service. After adding a > new service, the UI displays the ID, not the name. The service is created > but the client config remains in a warning state. > Message states > 'SSL Context Service' validated against > '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references > a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414512781
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
+return;
+}
+} else {
+// The character didn't match the expected one in the record
separator, reset the separator matcher
+// and check if it is the first character of the separator.
+indexIntoSeparator = 0;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+// This character is the beginning of the record
separator, keep it
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string
built so far
+return;
+}
+}
+}
Review Comment:
Instead of all the comments regarding reaching the end of a record, make a
method which conveys the intention of what you are trying to accomplish.
```
char nextChar = (char)code;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength) ) {
return;
}
} else {
// The character didn't match the expected one in the record separator,
reset the separator matcher
// and check if it is the first character of the separator.
indexIntoSeparator = 0;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength)
) {
return;
}
}
}
private boolean hasReachedEndOfRecord(int indexIntoSeparator, int
recordSeparatorLength) {
return indexIntoSeparator == recordSeparatorLength;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414512781
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
+return;
+}
+} else {
+// The character didn't match the expected one in the record
separator, reset the separator matcher
+// and check if it is the first character of the separator.
+indexIntoSeparator = 0;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+// This character is the beginning of the record
separator, keep it
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string
built so far
+return;
+}
+}
+}
Review Comment:
Instead of all the comments here, make a method which conveys the intention
of what you are trying to accomplish.
```
char nextChar = (char)code;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength) ) {
return;
}
} else {
// The character didn't match the expected one in the record separator,
reset the separator matcher
// and check if it is the first character of the separator.
indexIntoSeparator = 0;
if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
++indexIntoSeparator;
if (hasReachedEndOfRecord(indexIntoSeparator, recordSeparatorLength)
) {
return;
}
}
}
private boolean hasReachedEndOfRecord(int indexIntoSeparator, int
recordSeparatorLength) {
return indexIntoSeparator == recordSeparatorLength;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414467199
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
Review Comment:
```suggestion
// Short circuit as the matched separator indicates a
record has been read
```
Ditto for line 209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
exceptionfactory commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414496998
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PublishSlack.java:
##
@@ -0,0 +1,513 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack;
+
+import com.slack.api.bolt.App;
+import com.slack.api.bolt.AppConfig;
+import com.slack.api.methods.MethodsClient;
+import com.slack.api.methods.request.chat.ChatPostMessageRequest;
+import com.slack.api.methods.request.files.FilesUploadV2Request;
+import com.slack.api.methods.response.chat.ChatPostMessageResponse;
+import com.slack.api.methods.response.files.FilesUploadV2Response;
+import com.slack.api.model.File;
+import com.slack.api.model.File.ShareDetail;
+import com.slack.api.model.File.Shares;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.MultiProcessorUseCase;
+import org.apache.nifi.annotation.documentation.ProcessorConfiguration;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.documentation.UseCase;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.slack.consume.ConsumeSlackUtil;
+import org.apache.nifi.processors.slack.util.ChannelMapper;
+import org.apache.nifi.processors.slack.util.RateLimit;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.FormatUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("""
+Posts a message to the specified Slack channel. The content of the message
can be either a user-defined message that makes use of Expression Language or
+the contents of the FlowFile can be sent as the message. If sending a
user-defined message, the contents of the FlowFile may also be optionally
uploaded as
+a file attachment.
+""")
+@SeeAlso({ListenSlack.class, ConsumeSlack.class})
+@Tags({"slack", "conversation", "chat.postMessage", "social media", "team",
"text", "unstructured", "write", "upload", "send", "publish"})
+@WritesAttributes({
+@WritesAttribute(attribute = "slack.channel.id", description = "The ID of
the Slack Channel from which the messages were retrieved"),
+@WritesAttribute(attribute = "slack.ts", description = "The timestamp of
the slack messages that was sent; this is used by Slack as a unique identifier")
+})
+@DefaultSettings(yieldDuration = "3 sec")
+@UseCase(
+description = "Send specific text as a message to Slack, optionally
including the FlowFile's contents as an attached file.",
+configuration = """
+Set "Access Token" to the value of your Slack OAuth Access Token.
+Set "Channel" to the ID of the channel or the name of the channel
prefixed with the # symbol. For example, "C0123456789" or "#general".
+
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414495581
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PublishSlack.java:
##
@@ -0,0 +1,513 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack;
+
+import com.slack.api.bolt.App;
+import com.slack.api.bolt.AppConfig;
+import com.slack.api.methods.MethodsClient;
+import com.slack.api.methods.request.chat.ChatPostMessageRequest;
+import com.slack.api.methods.request.files.FilesUploadV2Request;
+import com.slack.api.methods.response.chat.ChatPostMessageResponse;
+import com.slack.api.methods.response.files.FilesUploadV2Response;
+import com.slack.api.model.File;
+import com.slack.api.model.File.ShareDetail;
+import com.slack.api.model.File.Shares;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.MultiProcessorUseCase;
+import org.apache.nifi.annotation.documentation.ProcessorConfiguration;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.documentation.UseCase;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.slack.consume.ConsumeSlackUtil;
+import org.apache.nifi.processors.slack.util.ChannelMapper;
+import org.apache.nifi.processors.slack.util.RateLimit;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.FormatUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("""
+Posts a message to the specified Slack channel. The content of the message
can be either a user-defined message that makes use of Expression Language or
+the contents of the FlowFile can be sent as the message. If sending a
user-defined message, the contents of the FlowFile may also be optionally
uploaded as
+a file attachment.
+""")
+@SeeAlso({ListenSlack.class, ConsumeSlack.class})
+@Tags({"slack", "conversation", "chat.postMessage", "social media", "team",
"text", "unstructured", "write", "upload", "send", "publish"})
+@WritesAttributes({
+@WritesAttribute(attribute = "slack.channel.id", description = "The ID of
the Slack Channel from which the messages were retrieved"),
+@WritesAttribute(attribute = "slack.ts", description = "The timestamp of
the slack messages that was sent; this is used by Slack as a unique identifier")
+})
+@DefaultSettings(yieldDuration = "3 sec")
+@UseCase(
+description = "Send specific text as a message to Slack, optionally
including the FlowFile's contents as an attached file.",
+configuration = """
+Set "Access Token" to the value of your Slack OAuth Access Token.
+Set "Channel" to the ID of the channel or the name of the channel
prefixed with the # symbol. For example, "C0123456789" or "#general".
+Set
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414488743
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/util/RateLimit.java:
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack.util;
+
+import org.apache.nifi.logging.ComponentLog;
+
+import java.util.Date;
+import java.util.concurrent.atomic.AtomicLong;
+
+public class RateLimit {
+private final AtomicLong nextRequestTime = new AtomicLong(0L);
+private final ComponentLog logger;
+
+public RateLimit(final ComponentLog logger) {
+this.logger = logger;
+}
+
+public void retryAfter(final int seconds) {
Review Comment:
Yeah, good call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 commented on code in PR #8120: URL: https://github.com/apache/nifi/pull/8120#discussion_r1414486316 ## nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/pom.xml: ## @@ -146,6 +142,11 @@ nifi-proxy-configuration-api test + +org.apache.nifi +nifi-api +compile Review Comment: Nope, not sure how that happened :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12469) Cannot configure registry client (Core Nifi)
[ https://issues.apache.org/jira/browse/NIFI-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793029#comment-17793029 ] David Handermann commented on NIFI-12469: - Thanks for reporting this issue [~wpkinzel]. Can you provide some additional context on the problem? Is this a new install of NiFi 2.0.0-M1, or was this an upgrade from an earlier version? Is the SSL Context Service enabled under Management Controller Services? > Cannot configure registry client (Core Nifi) > > > Key: NIFI-12469 > URL: https://issues.apache.org/jira/browse/NIFI-12469 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1 >Reporter: Bill Kinzel >Priority: Major > Attachments: nifi.JPG, stacktrace.txt > > > Nav = Nifi Settings > Registry Clients (Add) > There is an issue with configuring the SSL Context Service. After adding a > new service, the UI displays the ID, not the name. The service is created > but the client config remains in a warning state. > Message states > 'SSL Context Service' validated against > '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references > a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12469) Cannot configure registry client (Core Nifi)
[ https://issues.apache.org/jira/browse/NIFI-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12469: Affects Version/s: 2.0.0-M1 (was: 2.0.0) > Cannot configure registry client (Core Nifi) > > > Key: NIFI-12469 > URL: https://issues.apache.org/jira/browse/NIFI-12469 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1 >Reporter: Bill Kinzel >Priority: Major > Attachments: nifi.JPG, stacktrace.txt > > > Nav = Nifi Settings > Registry Clients (Add) > There is an issue with configuring the SSL Context Service. After adding a > new service, the UI displays the ID, not the name. The service is created > but the client config remains in a warning state. > Message states > 'SSL Context Service' validated against > '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references > a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12331: Added PublishSlack Processor [nifi]
exceptionfactory commented on code in PR #8120:
URL: https://github.com/apache/nifi/pull/8120#discussion_r1414460826
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/pom.xml:
##
@@ -146,6 +142,11 @@
nifi-proxy-configuration-api
test
+
+org.apache.nifi
+nifi-api
+compile
Review Comment:
Is there a reason for this change? The `nifi-api` should be provided.
##
nifi-nar-bundles/nifi-slack-bundle/nifi-slack-processors/src/main/java/org/apache/nifi/processors/slack/PublishSlack.java:
##
@@ -0,0 +1,513 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.slack;
+
+import com.slack.api.bolt.App;
+import com.slack.api.bolt.AppConfig;
+import com.slack.api.methods.MethodsClient;
+import com.slack.api.methods.request.chat.ChatPostMessageRequest;
+import com.slack.api.methods.request.files.FilesUploadV2Request;
+import com.slack.api.methods.response.chat.ChatPostMessageResponse;
+import com.slack.api.methods.response.files.FilesUploadV2Response;
+import com.slack.api.model.File;
+import com.slack.api.model.File.ShareDetail;
+import com.slack.api.model.File.Shares;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.MultiProcessorUseCase;
+import org.apache.nifi.annotation.documentation.ProcessorConfiguration;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.documentation.UseCase;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.slack.consume.ConsumeSlackUtil;
+import org.apache.nifi.processors.slack.util.ChannelMapper;
+import org.apache.nifi.processors.slack.util.RateLimit;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.FormatUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("""
+Posts a message to the specified Slack channel. The content of the message
can be either a user-defined message that makes use of Expression Language or
+the contents of the FlowFile can be sent as the message. If sending a
user-defined message, the contents of the FlowFile may also be optionally
uploaded as
+a file attachment.
+""")
+@SeeAlso({ListenSlack.class, ConsumeSlack.class})
+@Tags({"slack", "conversation", "chat.postMessage", "social media", "team",
"text", "unstructured", "write", "upload", "send", "publish"})
+@WritesAttributes({
+@WritesAttribute(attribute = "slack.channel.id", description = "The ID of
the Slack Channel from which the messages were retrieved"),
+@WritesAttribute(attribute = "slack.ts", description = "The timestamp of
the slack messages that was sent; this is used by Slack as a unique identifier")
+})
+@DefaultSettings(yieldDuration = "3 sec")
+@UseCase(
+description = "Send specific text as a
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414467199
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
Review Comment:
```suggestion
// Short circuit as the matched separator indicates a
record has been read
```
Ditto for line 209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414467199
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
Review Comment:
```suggestion
// Short circuit as the matched separator indicates a
record has been read
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-8932: Add capability to skip first N rows in CSVReader [nifi]
dan-s1 commented on code in PR #7952:
URL: https://github.com/apache/nifi/pull/7952#discussion_r1414467199
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java:
##
@@ -158,4 +180,46 @@ protected String trim(String value) {
public RecordSchema getSchema() {
return schema;
}
+
+/**
+ * This method searches using the specified Reader character-by-character
until the
+ * record separator is found.
+ * @param reader the Reader providing the input
+ * @param recordSeparator the String specifying the end of a record in the
input
+ * @throws IOException if an error occurs during reading, including not
finding the record separator in the input
+ */
+protected void readNextRecord(Reader reader, String recordSeparator)
throws IOException {
+int indexIntoSeparator = 0;
+int recordSeparatorLength = recordSeparator.length();
+int code = reader.read();
+while (code != -1) {
+char nextChar = (char)code;
+if (recordSeparator.charAt(indexIntoSeparator) == nextChar) {
+if (++indexIntoSeparator == recordSeparatorLength) {
+// We have matched the separator, return the string built
so far
Review Comment:
```suggestion
// Short circuit as the separator indicates a record has
been read
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
exceptionfactory commented on PR #8104: URL: https://github.com/apache/nifi/pull/8104#issuecomment-1839432708 Thanks for the review @joewitt, I pushed an update based on the feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
exceptionfactory commented on code in PR #8104: URL: https://github.com/apache/nifi/pull/8104#discussion_r1414458619 ## nifi-assembly/pom.xml: ## @@ -906,11 +918,6 @@ language governing permissions and limitations under the License. --> 2.0.0-SNAPSHOT nar - Review Comment: Thanks for the feedback. Although loading native libraries may not be necessary for most libraries, this makes sense for the edge cases, and support for the aspect implementation also available. The latest commit restores the AspectJ references. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
exceptionfactory commented on code in PR #8104:
URL: https://github.com/apache/nifi/pull/8104#discussion_r1414440831
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf:
##
@@ -45,75 +42,14 @@ java.arg.3=-Xmx${nifi.jvm.heap.max}
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol
-#Set headless mode by default
-java.arg.14=-Djava.awt.headless=true
+# Enable Headless mode to avoid HeadlessException with Java AWT libraries
+java.arg.headless=-Djava.awt.headless=true
+
+# Configure Apache Curator connection logging for Apache ZooKeeper to avoid
excessive ERROR messages
+java.arg.curatorLogOnlyFirstConnectionIssue=-Dcurator-log-only-first-connection-issue-as-error-level=true
# Root key in hexadecimal format for encrypted sensitive configuration values
nifi.bootstrap.sensitive.key=
-# Sensitive Property Provider configuration
-
-# HashiCorp Vault Sensitive Property Providers
-#nifi.bootstrap.protection.hashicorp.vault.conf=./conf/bootstrap-hashicorp-vault.conf
-
-# AWS Sensitive Property Providers
-#nifi.bootstrap.protection.aws.conf=./conf/bootstrap-aws.conf
-
-# Azure Key Vault Sensitive Property Providers
-#nifi.bootstrap.protection.azure.keyvault.conf=./conf/bootstrap-azure.conf
-
-# GCP KMS Sensitive Property Providers
-#nifi.bootstrap.protection.gcp.kms.conf=./conf/bootstrap-gcp.conf
-
-# Sets the provider of SecureRandom to /dev/urandom to prevent blocking on VMs
-java.arg.15=-Djava.security.egd=file:/dev/urandom
-
-# Requires JAAS to use only the provided JAAS configuration to authenticate a
Subject, without using any "fallback" methods (such as prompting for
username/password)
-# Please see
https://docs.oracle.com/en/java/javase/17/security/single-sign-using-kerberos-java1.html,
section "EXCEPTIONS TO THE MODEL"
-java.arg.16=-Djavax.security.auth.useSubjectCredsOnly=true
Review Comment:
That's a good point, it is common enough that it is a useful default. I
reverted the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
exceptionfactory commented on code in PR #8104:
URL: https://github.com/apache/nifi/pull/8104#discussion_r1414440489
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf:
##
@@ -45,75 +42,14 @@ java.arg.3=-Xmx${nifi.jvm.heap.max}
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol
-#Set headless mode by default
-java.arg.14=-Djava.awt.headless=true
+# Enable Headless mode to avoid HeadlessException with Java AWT libraries
+java.arg.headless=-Djava.awt.headless=true
+
+# Configure Apache Curator connection logging for Apache ZooKeeper to avoid
excessive ERROR messages
+java.arg.curatorLogOnlyFirstConnectionIssue=-Dcurator-log-only-first-connection-issue-as-error-level=true
# Root key in hexadecimal format for encrypted sensitive configuration values
nifi.bootstrap.sensitive.key=
-# Sensitive Property Provider configuration
-
-# HashiCorp Vault Sensitive Property Providers
Review Comment:
Yes, I agree they should be retained for now. Restored in the most commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] NIFI-12331: Added PublishSlack Processor [nifi]
markap14 opened a new pull request, #8120: URL: https://github.com/apache/nifi/pull/8120 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update Dockerfile for alpine v3 / liberica v21 [nifi]
joewitt commented on PR #8111: URL: https://github.com/apache/nifi/pull/8111#issuecomment-1839384718 Ahhh that is a great point @exceptionfactory. It might make sense to support/include a non python Java 2.0 based image on Alpine. That would end up a good bit smaller too. Something to consider as an alternative JIRA/PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12450) Deprecate Bootstrap Notification Services for Removal
[ https://issues.apache.org/jira/browse/NIFI-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793021#comment-17793021 ] ASF subversion and git services commented on NIFI-12450: Commit 60fe7949067764f27ae898a7335aed569f64097c in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=60fe794906 ] NIFI-12450 This closes #8103. Added Deprecation Warning for Bootstrap Notifications Signed-off-by: Joseph Witt > Deprecate Bootstrap Notification Services for Removal > - > > Key: NIFI-12450 > URL: https://issues.apache.org/jira/browse/NIFI-12450 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The NiFi bootstrap process includes optional notification services to send > email messages or call HTTP resources to indicate when NiFi is stopped or > started. > Although this level of notification is useful in some circumstances, it does > not provide sufficient capabilities to warrant continued maintenance. Various > service monitoring tools provide more robust health and status checking > capabilities, and much more flexible notification options. For these reasons, > the Notification Services should be deprecated on the version 1 support > branch for subsequent removal in the next major release version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12450) Deprecate Bootstrap Notification Services for Removal
[ https://issues.apache.org/jira/browse/NIFI-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12450: Fix Version/s: 1.25.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Deprecate Bootstrap Notification Services for Removal > - > > Key: NIFI-12450 > URL: https://issues.apache.org/jira/browse/NIFI-12450 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.25.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The NiFi bootstrap process includes optional notification services to send > email messages or call HTTP resources to indicate when NiFi is stopped or > started. > Although this level of notification is useful in some circumstances, it does > not provide sufficient capabilities to warrant continued maintenance. Various > service monitoring tools provide more robust health and status checking > capabilities, and much more flexible notification options. For these reasons, > the Notification Services should be deprecated on the version 1 support > branch for subsequent removal in the next major release version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12450 Add Deprecation Warning for Bootstrap Notifications [nifi]
joewitt closed pull request #8103: NIFI-12450 Add Deprecation Warning for Bootstrap Notifications URL: https://github.com/apache/nifi/pull/8103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
joewitt commented on code in PR #8104:
URL: https://github.com/apache/nifi/pull/8104#discussion_r1414418481
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf:
##
@@ -45,75 +42,14 @@ java.arg.3=-Xmx${nifi.jvm.heap.max}
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol
-#Set headless mode by default
-java.arg.14=-Djava.awt.headless=true
+# Enable Headless mode to avoid HeadlessException with Java AWT libraries
+java.arg.headless=-Djava.awt.headless=true
+
+# Configure Apache Curator connection logging for Apache ZooKeeper to avoid
excessive ERROR messages
+java.arg.curatorLogOnlyFirstConnectionIssue=-Dcurator-log-only-first-connection-issue-as-error-level=true
# Root key in hexadecimal format for encrypted sensitive configuration values
nifi.bootstrap.sensitive.key=
-# Sensitive Property Provider configuration
-
-# HashiCorp Vault Sensitive Property Providers
-#nifi.bootstrap.protection.hashicorp.vault.conf=./conf/bootstrap-hashicorp-vault.conf
-
-# AWS Sensitive Property Providers
-#nifi.bootstrap.protection.aws.conf=./conf/bootstrap-aws.conf
-
-# Azure Key Vault Sensitive Property Providers
-#nifi.bootstrap.protection.azure.keyvault.conf=./conf/bootstrap-azure.conf
-
-# GCP KMS Sensitive Property Providers
-#nifi.bootstrap.protection.gcp.kms.conf=./conf/bootstrap-gcp.conf
-
-# Sets the provider of SecureRandom to /dev/urandom to prevent blocking on VMs
-java.arg.15=-Djava.security.egd=file:/dev/urandom
-
-# Requires JAAS to use only the provided JAAS configuration to authenticate a
Subject, without using any "fallback" methods (such as prompting for
username/password)
-# Please see
https://docs.oracle.com/en/java/javase/17/security/single-sign-using-kerberos-java1.html,
section "EXCEPTIONS TO THE MODEL"
-java.arg.16=-Djavax.security.auth.useSubjectCredsOnly=true
Review Comment:
@exceptionfactory I am not aware of any situation in NiFi usage whereby we'd
ever want someone with a Kerberos setup to allow any prompting for user input
at the command line. This prevents that. I recommend we retain this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12469) Cannot configure registry client (Core Nifi)
[ https://issues.apache.org/jira/browse/NIFI-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Kinzel updated NIFI-12469: --- Attachment: stacktrace.txt > Cannot configure registry client (Core Nifi) > > > Key: NIFI-12469 > URL: https://issues.apache.org/jira/browse/NIFI-12469 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0 >Reporter: Bill Kinzel >Priority: Major > Attachments: nifi.JPG, stacktrace.txt > > > Nav = Nifi Settings > Registry Clients (Add) > There is an issue with configuring the SSL Context Service. After adding a > new service, the UI displays the ID, not the name. The service is created > but the client config remains in a warning state. > Message states > 'SSL Context Service' validated against > '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references > a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
joewitt commented on code in PR #8104:
URL: https://github.com/apache/nifi/pull/8104#discussion_r1414413340
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf:
##
@@ -45,75 +42,14 @@ java.arg.3=-Xmx${nifi.jvm.heap.max}
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol
-#Set headless mode by default
-java.arg.14=-Djava.awt.headless=true
+# Enable Headless mode to avoid HeadlessException with Java AWT libraries
+java.arg.headless=-Djava.awt.headless=true
+
+# Configure Apache Curator connection logging for Apache ZooKeeper to avoid
excessive ERROR messages
+java.arg.curatorLogOnlyFirstConnectionIssue=-Dcurator-log-only-first-connection-issue-as-error-level=true
# Root key in hexadecimal format for encrypted sensitive configuration values
nifi.bootstrap.sensitive.key=
-# Sensitive Property Provider configuration
-
-# HashiCorp Vault Sensitive Property Providers
Review Comment:
@exceptionfactory This seems out of the scope of this changeset related to
notification services. Same with the other sensitive prop provider updates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-12469) Cannot configure registry client (Core Nifi)
Bill Kinzel created NIFI-12469: -- Summary: Cannot configure registry client (Core Nifi) Key: NIFI-12469 URL: https://issues.apache.org/jira/browse/NIFI-12469 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 2.0.0 Reporter: Bill Kinzel Attachments: nifi.JPG Nav = Nifi Settings > Registry Clients (Add) There is an issue with configuring the SSL Context Service. After adding a new service, the UI displays the ID, not the name. The service is created but the client config remains in a warning state. Message states 'SSL Context Service' validated against '018c1010-305c-1756-5807-59fc1c828af4' is invalid because Property references a Controller Service that does not exist -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12451 Remove Bootstrap Notification Services [nifi]
joewitt commented on code in PR #8104: URL: https://github.com/apache/nifi/pull/8104#discussion_r1414400437 ## nifi-assembly/pom.xml: ## @@ -906,11 +918,6 @@ language governing permissions and limitations under the License. --> 2.0.0-SNAPSHOT nar - Review Comment: @exceptionfactory I think we need to keep the aspectj references here and all others below as that is how our native lib loading mechanism works and that is a powerful/key feature for any nars that might need to load native libs and still benefit from classloader isolation even in the event of native libs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12452 Improve support for DescribedValue [nifi]
EndzeitBegins commented on PR #8102: URL: https://github.com/apache/nifi/pull/8102#issuecomment-1839332687 Yes, that's what I was concerned about and why I didn't use an overload in the first draft. If we do not backport this change, maybe we can introduce this as a breaking change for 2.x and add an deprecation for calling the function with `null` in 1.x? As far as I understand there is no reason to call it with `null` anyway, as it does nothing. The only use case I can image where calling it with `null` makes sense is to reset a default value copied from a different PropertyDescriptor. However, that isn't what's happening. Maybe we should add a `clearDefaultValue` for that, similar to the existing methods for other fields. While accepting an `Object` avoids the breaking change, I think it makes the interface less easy to understand. Personally I prefer strongly (tightly) types APIs where applicable. I would prefer the deprecation approach. If that's not feasible I'd rather go with the different name than accepting any type, but that's just my gut feeling. What are your thoughts on this @markap14? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12468) Xodus Flow Configuration History Environment Inoperative after Stream Closed
[
https://issues.apache.org/jira/browse/NIFI-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793013#comment-17793013
]
David Handermann commented on NIFI-12468:
-
Thanks for the reply [~mosermw].
Yes, it is helpful to have this captured for future reference of potentially
similar scenarios. I adjusted the title of the issue to mention Environment
Inoperative and Stream Closed to highlight the particulars.
If this happen due to something like running out of disk space, the appropriate
action probably would be to stop NiFi and evaluate other issues.
As the Xodus implementation already handles recovery on startup, this is worth
noting as background for potential improvement if we have similar scenarios
from general usage.
> Xodus Flow Configuration History Environment Inoperative after Stream Closed
>
>
> Key: NIFI-12468
> URL: https://issues.apache.org/jira/browse/NIFI-12468
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M1
>Reporter: Michael W Moser
>Priority: Minor
>
> I caused Xodus to timeout writing a transation to flow configuration history.
> Thereafter, every operation that would update flow configuration history, or
> trying to view flow configuration history, continues to throw an exception.
> The UI shows "An unexpected error has occurred. Please check the logs for
> additional details" based on a 500 response from the REST API
> /nifi-api/flow/history
> I had to restart NiFi to recover. Hopefully, this code can be improved to
> detect exceptions and recover on its own.
> I caused the initial problem by setting a breakpoint while a debugger was
> attached to NiFi. I dragged a new Process Group onto the graph from a flow
> definition file and let it sit at the breakpoint for a while.
> {noformat}
> 2023-12-04 16:32:00,448 ERROR [NiFi Web Server-466]
> jetbrains.exodus.env.EnvironmentImpl Failed to flush transaction
> java.io.IOException: Stream Closed
> at java.base/java.io.RandomAccessFile.length0(Native Method)
> at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
> at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
> at jetbrains.exodus.io.FileDataWriter.write(FileDataWriter.kt:59)
> at
> jetbrains.exodus.log.BufferedDataWriter.writePage(BufferedDataWriter.java:264)
> at
> jetbrains.exodus.log.BufferedDataWriter.flush(BufferedDataWriter.java:176)
> at jetbrains.exodus.log.Log.flush(Log.kt:631)
> at jetbrains.exodus.log.Log.flush$default(Log.kt:630)
> at jetbrains.exodus.log.Log.flush(Log.kt)
> at
> jetbrains.exodus.env.EnvironmentImpl.flushTransaction(EnvironmentImpl.java:731)
> at
> jetbrains.exodus.env.EnvironmentImpl.commitTransaction(EnvironmentImpl.java:684)
> at
> jetbrains.exodus.env.ReadWriteTransaction.commit(ReadWriteTransaction.java:96)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.doCommit(PersistentStoreTransaction.java:192)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.commit(PersistentStoreTransaction.java:180)
> at
> jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:720)
> at
> org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
> at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
> at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
> at
> org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
> at
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> {noformat}
> Afterwards, any attempt to read or write from Xodus produces an exception
> similar to this:
> {noformat}
> 2023-12-04 16:32:53,111 WARN [NiFi Web Server-371]
> o.apache.nifi.audit.ProcessGroupAuditor Unable to record actions:
> jetbrains.exodus.ExodusException: Environment is inoperative
> at
> jetbrains.exodus.ExodusException.toExodusException(ExodusException.java:49)
> at
> jetbrains.exodus.env.EnvironmentImpl.checkIsOperative(EnvironmentImpl.java:1069)
> at
> jetbrains.exodus.env.EnvironmentImpl.beginTransaction(EnvironmentImpl.java:613)
> at
> jetbrains.exodus.env.EnvironmentImpl.beginExclusiveTransaction(EnvironmentImpl.java:264)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.(PersistentStoreTransaction.java:130)
> at
> jetbrains.exodus.entitystore.PersistentEntityStoreImpl.beginExclusiveTransaction(PersistentEntityStoreImpl.java:539)
> at
>
