[jira] [Commented] (MINIFICPP-1999) Persistent state of processors
[
https://issues.apache.org/jira/browse/MINIFICPP-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641127#comment-17641127
]
Gábor Gyimesi commented on MINIFICPP-1999:
--
"by passing the command and the name of the processor (it's strange that you
can't pass its uid))"
Regarding this comment UUID can also be used to start/stop a component in the
latest version of MiNiFi. This was implemented in the following change:
[https://github.com/apache/nifi-minifi-cpp/pull/1388] It will be available in
the 0.13.0 release.
> Persistent state of processors
> --
>
> Key: MINIFICPP-1999
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1999
> Project: Apache NiFi MiNiFi C++
> Issue Type: New Feature
>Affects Versions: 0.12.0
>Reporter: Kondakov Artem
>Priority: Major
> Fix For: 0.14.0
>
>
> Hi
> In apache/nifi each processor can be started and stopped. It is also possible
> to activate and deactivate. In this case, the state of the processors is
> saved when the application is restarted.
> minifi-cpp also has the ability to start and stop individual processors using
> the C2 protocol or minificontroller (by passing the command and the name of
> the processor (it's strange that you can't pass its uid))
> However, after restarting minifi all the same, all processors start, while
> the permanent state store is configured
>
> {color:#172b4d}nifi.state.management.provider.local.path=./repositories/componentstate{color}
> {color:#00875a}## To make the default state storage persist every state
> change, set this to true{color}
> {color:#00875a}## this comes at a performance penalty, but makes sure no
> state is lost even on unclean shutdowns{color}
> {color:#172b4d}nifi.state.management.provider.local.always.persist=true{color}
> {color:#00875a}## To change the frequency at which the default state storage
> is persisted, modify the following{color}
> {color:#172b4d}nifi.state.management.provider.local.auto.persistence.interval=1min{color}
> I think it would be very helpful
> - support state storage for processors (active/inactive)
> - support changing and storing state also for other types of nodes (input
> ports, output ports, funnels, process groups)
> - adding the "enabled" property to the properties of processors in yaml (and
> other types of nodes) by analogy with Apache / nifi,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1940) Remove unused parameter from FlowConfiguration constructor
[ https://issues.apache.org/jira/browse/MINIFICPP-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1940: - Status: Patch Available (was: In Progress) > Remove unused parameter from FlowConfiguration constructor > -- > > Key: MINIFICPP-1940 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1940 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1987) Configuring a metrics for Processors for C2 Protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1987: - Status: Patch Available (was: In Progress) > Configuring a metrics for Processors for C2 Protocol > > > Key: MINIFICPP-1987 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1987 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.14.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hello. > I tried to setup receiving metrics using the C2 protocol for my processors. > The documentation describes that you must explicitly set the metric classes, > for example like this > nifi.c2.root.class.definitions.metrics.metrics.processorMetrics.classes=ListenHTTPMetrics,PostElasticsearchMetrics,InvokeHTTPMetrics,BinFilesMetrics,MergeContentMetrics... > It turns out a complex configuration, if I want to receive data on all my > processors, I need to update the list of these classes every time if I add a > new or remove type of processor. > If I try to configure classes for all possible processors, then I get 50+ > errors in the logs, , because not all processor types are involved, for > example > "No metric defined for PublishKafkaMetrics" > "No metric defined for ConsumeKafkaMetrics" > etc... > It seems to me that it is worth making one universal metric class for all > processes, or lower the error logging level when searching for a metric class > to warning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1995) Add configuring path for flowfile_checkpoint directory
[ https://issues.apache.org/jira/browse/MINIFICPP-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1995: - Status: Patch Available (was: In Progress) > Add configuring path for flowfile_checkpoint directory > -- > > Key: MINIFICPP-1995 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1995 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > In the minifi.properties file, there is no way to set the directory path for > flowfile_checkpoint, similar to > * nifi.provenance.repository.directory.default > * nifi.flowfile.repository.directory.default > * nifi.state.management.provider.local.path -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1983) Mount minifi.properties file in docker tests
[ https://issues.apache.org/jira/browse/MINIFICPP-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1983: - Status: Patch Available (was: In Progress) > Mount minifi.properties file in docker tests > > > Key: MINIFICPP-1983 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1983 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > There are multiple versions of the minifi image in the docker tests that only > differ in the used minifi.properties file. Different config files should be > mounted instead of creating separate images for them. This way we could also > disable options like C2 in tests where they are not needed. > The config.yml file handling could also be refactored to be mounted directly > instead of mounting it to the /tmp directory and copying to the minifi/conf > directory at start. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1993) Minifi crashing while getting flow-file content in python processor
[ https://issues.apache.org/jira/browse/MINIFICPP-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1993: - Status: Patch Available (was: In Progress) > Minifi crashing while getting flow-file content in python processor > --- > > Key: MINIFICPP-1993 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1993 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Attachments: conf.zip, image-2022-11-22-19-46-11-969.png, > image-2022-11-22-19-48-08-443.png, minifi-python.zip > > Time Spent: 10m > Remaining Estimate: 0h > > Hello. > We built version 0.12.0 with the ability to execute python scripts as > processors > Made a simple process (see attachment) - calling and getting data using > invokehttp get. After that, a custom python processor is executed (see > attachment). > Inside this python script, there is an attempt to get the contents of > flow-file. At this point, an error occurs that stops the entire minifi > application. At this code line > !image-2022-11-22-19-46-11-969.png! > There is a log file where you can see that the error occurs on this line of > python code. > !image-2022-11-22-19-48-08-443.png! > At the same time, wrapping the python code in try / except does not give > anything, apparently the fall occurs somewhere outside the python runtime > After some time debugging, it turned out that this behavior only occurs when > a flow-file is passed from the InvokeHttp processor through the relationship > type success > In this case, the transmitted content is empty (you can connect PutFile and > make sure that an empty file will be saved) > If the relationship type is set to response, then everything works as > expected without minifi crashing. > The expected behavior is that minifi will not suddenly stop if an attempt is > made to get the contents of an empty flow-file > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-1993) Minifi crashing while getting flow-file content in python processor
[ https://issues.apache.org/jira/browse/MINIFICPP-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641122#comment-17641122 ] Gábor Gyimesi commented on MINIFICPP-1993: -- PR with the fix available at https://github.com/apache/nifi-minifi-cpp/pull/1463 > Minifi crashing while getting flow-file content in python processor > --- > > Key: MINIFICPP-1993 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1993 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Attachments: conf.zip, image-2022-11-22-19-46-11-969.png, > image-2022-11-22-19-48-08-443.png, minifi-python.zip > > Time Spent: 10m > Remaining Estimate: 0h > > Hello. > We built version 0.12.0 with the ability to execute python scripts as > processors > Made a simple process (see attachment) - calling and getting data using > invokehttp get. After that, a custom python processor is executed (see > attachment). > Inside this python script, there is an attempt to get the contents of > flow-file. At this point, an error occurs that stops the entire minifi > application. At this code line > !image-2022-11-22-19-46-11-969.png! > There is a log file where you can see that the error occurs on this line of > python code. > !image-2022-11-22-19-48-08-443.png! > At the same time, wrapping the python code in try / except does not give > anything, apparently the fall occurs somewhere outside the python runtime > After some time debugging, it turned out that this behavior only occurs when > a flow-file is passed from the InvokeHttp processor through the relationship > type success > In this case, the transmitted content is empty (you can connect PutFile and > make sure that an empty file will be saved) > If the relationship type is set to response, then everything works as > expected without minifi crashing. > The expected behavior is that minifi will not suddenly stop if an attempt is > made to get the contents of an empty flow-file > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1990) Transferring FlowFiles in Python Processors
[ https://issues.apache.org/jira/browse/MINIFICPP-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-1990: - Status: Patch Available (was: In Progress) > Transferring FlowFiles in Python Processors > --- > > Key: MINIFICPP-1990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.14.0 > > Attachments: image-2022-11-23-20-03-09-400.png, > image-2022-11-23-20-03-53-653.png > > Time Spent: 10m > Remaining Estimate: 0h > > Hello. > There is a problem with the processing of flow-files in python processors, > which are connected using the setting > nifi.python.processor.dir=./minifi-python/ > If you need to split one file into several, then you have to create new files > in the Python processor and be sure to transfer the original one to > REL_FAILURE. > Otherwise, if I do not pass the original file to REL_FAILURE, then the > processing result is rolled back and nothing happens > It seems to me that it is necessary to make it possible for Python processors > to add various relations, and not just REL_FAILURE REL_SUCCESS. > Or add an additional possible relationship REL_ORIGIN to which you can pass > the original file > That. if you look at the metrics for these processors, you see > TransferredToFaillure > 0, although the processors processed without errors > Another additional possibility is to add the session.remove (flowfile ) > method (as it is done for example in the processor ExecuteSQL). Then it will > be possible to exclude incoming files from further processing > !image-2022-11-23-20-03-53-653.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-1990) Transferring FlowFiles in Python Processors
[ https://issues.apache.org/jira/browse/MINIFICPP-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641121#comment-17641121 ] Gábor Gyimesi commented on MINIFICPP-1990: -- PR with the change available at: https://github.com/apache/nifi-minifi-cpp/pull/1462 > Transferring FlowFiles in Python Processors > --- > > Key: MINIFICPP-1990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.14.0 > > Attachments: image-2022-11-23-20-03-09-400.png, > image-2022-11-23-20-03-53-653.png > > Time Spent: 10m > Remaining Estimate: 0h > > Hello. > There is a problem with the processing of flow-files in python processors, > which are connected using the setting > nifi.python.processor.dir=./minifi-python/ > If you need to split one file into several, then you have to create new files > in the Python processor and be sure to transfer the original one to > REL_FAILURE. > Otherwise, if I do not pass the original file to REL_FAILURE, then the > processing result is rolled back and nothing happens > It seems to me that it is necessary to make it possible for Python processors > to add various relations, and not just REL_FAILURE REL_SUCCESS. > Or add an additional possible relationship REL_ORIGIN to which you can pass > the original file > That. if you look at the metrics for these processors, you see > TransferredToFaillure > 0, although the processors processed without errors > Another additional possibility is to add the session.remove (flowfile ) > method (as it is done for example in the processor ExecuteSQL). Then it will > be possible to exclude incoming files from further processing > !image-2022-11-23-20-03-53-653.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ZhewenFu commented on a diff in pull request #6596: NIFI-10717 fix inconsistent tests
ZhewenFu commented on code in PR #6596:
URL: https://github.com/apache/nifi/pull/6596#discussion_r1035539697
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/inheritance/MissingComponentsCheck.java:
##
@@ -34,15 +35,15 @@ public FlowInheritability checkInheritability(final
DataFlow existingFlow, final
existingMissingComponents.removeAll(proposedFlow.getMissingComponents());
if (existingMissingComponents.size() > 0) {
-final String missingIds =
StringUtils.join(existingMissingComponents, ",");
+final String missingIds =
StringUtils.join(existingMissingComponents.stream().sorted().collect(Collectors.toList()),
",");
Review Comment:
Hi, the current fix of using LinkedHashSet only fully solves the first test,
and the second test is still flaky so I left the sorting here. I have tried to
replace `HashSet` with `LinkedHashSet` on all the places where we check
`MissingComponenets` but the second test is still flaky in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] Kerr0220 commented on pull request #6507: NIFI-10622 Fix Flaky Test
Kerr0220 commented on PR #6507: URL: https://github.com/apache/nifi/pull/6507#issuecomment-133145 > Yes, sorry I'm still working on finding the root cause of this problem. I would appreciate it if you could give me more time on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6504: NIFI-10618: Add Asana connector
turcsanyip commented on code in PR #6504:
URL: https://github.com/apache/nifi/pull/6504#discussion_r1035214769
##
nifi-nar-bundles/nifi-asana-bundle/nifi-asana-services-api/src/main/java/org/apache/nifi/controller/asana/AsanaClient.java:
##
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller.asana;
+
+import com.asana.models.Attachment;
+import com.asana.models.Project;
+import com.asana.models.ProjectMembership;
+import com.asana.models.ProjectStatus;
+import com.asana.models.Section;
+import com.asana.models.Story;
+import com.asana.models.Tag;
+import com.asana.models.Task;
+import com.asana.models.Team;
+import com.asana.models.User;
+
+import java.util.Map;
+
+/**
+ * This interface represents a client to Asana REST server, with some basic
filtering options built in.
+ */
+public interface AsanaClient {
+/**
+ * Find & retrieve an Asana project by its name. If multiple projects
match, returns the first.
+ * If there is no match, then {@link RuntimeException} is thrown. Note
that constant ordering
Review Comment:
```suggestion
* If there is no match, then {@link AsanaClientException} is thrown.
Note that constant ordering
```
Also in the javadoc of `getTeamByName()` and `getSectionByName()`.
##
nifi-nar-bundles/nifi-asana-bundle/nifi-asana-services-api-nar/src/main/resources/META-INF/NOTICE:
##
@@ -0,0 +1,108 @@
+nifi-asana-services-api-nar
+Copyright 2015-2022 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+===
+Apache Software License v2
+===
+
+The following binary components are provided under the Apache Software License
v2
+
+ (ASLv2) Apache Commons IO
+The following NOTICE information applies:
+ Apache Commons IO
+ Copyright 2002-2022 The Apache Software Foundation
Review Comment:
Thanks for updating the `NOTICE` file, overall it looks good!
`Commons IO` is not present in the services-api-nar so it should be removed.
I forgot to mention earlier but the entries need to be added in the assembly
nar as well (`nifi-assembly/NOTICE`). This "global" NOTICE file is kind of a
union of all entries in the nar.
Could you please copy those items to that NOTICE file which are not present
there yet?
##
nifi-nar-bundles/nifi-asana-bundle/nifi-asana-processors/src/main/java/org/apache/nifi/processors/asana/GetAsanaObject.java:
##
@@ -0,0 +1,393 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.asana;
+
+import static java.lang.String.format;
+import static java.util.Collections.singletonMap;
+import static java.util.stream.Collectors.groupingBy;
+import static java.util.stream.Collectors.joining;
+import static org.apache.commons.collections4.ListUtils.partition;
+import static
org.apache.nifi.processors.asana.AsanaObjectType.AV_COLLECT_PROJECT_EVENTS;
+import static
org.apache.nifi.processors.asana.AsanaObjectType.AV_COLLECT_PROJECT_MEMBERS;
+import static
org.apache.nifi.processors.asana.AsanaObjectType.AV_COLLECT_PROJECT_STATUS_ATTACHMENTS;
+import static
org.apache.nifi.processors.asana.AsanaObjectType.AV_COLLECT_PROJECT_STATUS_UPDATES;
+import static
[jira] [Updated] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-10901: - Status: Patch Available (was: In Progress) https://github.com/apache/nifi/pull/6731/ > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] takraj commented on pull request #6504: NIFI-10618: Add Asana connector
takraj commented on PR #6504: URL: https://github.com/apache/nifi/pull/6504#issuecomment-1331352294 Rebased & squashed. Versions bumped to 1.20.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-maven] kevdoran commented on pull request #18: Bump checkstyle from 6.5 to 8.29
kevdoran commented on PR #18: URL: https://github.com/apache/nifi-maven/pull/18#issuecomment-1331316223 Declined in favor of #24 which updates to a more recent version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-maven] dependabot[bot] commented on pull request #18: Bump checkstyle from 6.5 to 8.29
dependabot[bot] commented on PR #18: URL: https://github.com/apache/nifi-maven/pull/18#issuecomment-1331316288 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-maven] kevdoran closed pull request #18: Bump checkstyle from 6.5 to 8.29
kevdoran closed pull request #18: Bump checkstyle from 6.5 to 8.29 URL: https://github.com/apache/nifi-maven/pull/18 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10906) Release NiFi NAR Maven Plugin 1.4.0
Kevin Doran created NIFI-10906: -- Summary: Release NiFi NAR Maven Plugin 1.4.0 Key: NIFI-10906 URL: https://issues.apache.org/jira/browse/NIFI-10906 Project: Apache NiFi Issue Type: Task Reporter: Kevin Doran Fix For: nifi-nar-maven-plugin-1.4.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10905) Bump NAR Maven Plugin checkstyle version
[ https://issues.apache.org/jira/browse/NIFI-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-10905: --- Status: Patch Available (was: Open) > Bump NAR Maven Plugin checkstyle version > > > Key: NIFI-10905 > URL: https://issues.apache.org/jira/browse/NIFI-10905 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: nifi-nar-maven-plugin-1.3.6 > > Time Spent: 10m > Remaining Estimate: 0h > > Update checkstyle to match NiFi (v9.3, which is the last version that > supports Java 8) > also externalize checkstyle rules into standalone xml file -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10904) Changing Font Color for Dropdown Menu When Selecting Controller Services
[ https://issues.apache.org/jira/browse/NIFI-10904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anson Yu reassigned NIFI-10904: --- Assignee: Anson Yu > Changing Font Color for Dropdown Menu When Selecting Controller Services > > > Key: NIFI-10904 > URL: https://issues.apache.org/jira/browse/NIFI-10904 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Anson Yu >Assignee: Anson Yu >Priority: Trivial > Attachments: GrayOut.PNG > > > When I was adding a controller service to a processor (such as > ConvertRecords) via the Properties tab in it's configurations, I noticed the > color of the base options in the dropdown containing "Reference parameter..." > & "Create new service..." used a grey color that gave the impression of it > being unavailable on first glance, even though were selectable options. I > think it would be better if the default color was something else. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-maven] kevdoran opened a new pull request, #24: NIFI-10905 Update checkstyle to v9.3
kevdoran opened a new pull request, #24: URL: https://github.com/apache/nifi-maven/pull/24 - Also update checkstyle plugin version - Also externalize checkstyle rules from pom file to standalone xml -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10905) Bump NAR Maven Plugin checkstyle version
Kevin Doran created NIFI-10905: -- Summary: Bump NAR Maven Plugin checkstyle version Key: NIFI-10905 URL: https://issues.apache.org/jira/browse/NIFI-10905 Project: Apache NiFi Issue Type: Improvement Reporter: Kevin Doran Assignee: Kevin Doran Fix For: nifi-nar-maven-plugin-1.3.6 Update checkstyle to match NiFi (v9.3, which is the last version that supports Java 8) also externalize checkstyle rules into standalone xml file -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog opened a new pull request, #6732: NIFI-10903 - Updated the registry pom with a fix for the build format…
thenatog opened a new pull request, #6732: URL: https://github.com/apache/nifi/pull/6732 …s, and now builds dir by default alongside the zip. # Summary [NIFI-10903](https://issues.apache.org/jira/browse/NIFI-10903) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10903) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-10903` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-10903` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6507: NIFI-10622 Fix Flaky Test
exceptionfactory commented on PR #6507: URL: https://github.com/apache/nifi/pull/6507#issuecomment-1331300066 @Kerr0220 Just checking, are you still planning on making changes to this PR to address the concerns? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10325) make timestamp independant from builders' timezone
[ https://issues.apache.org/jira/browse/NIFI-10325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran resolved NIFI-10325. Resolution: Fixed > make timestamp independant from builders' timezone > -- > > Key: NIFI-10325 > URL: https://issues.apache.org/jira/browse/NIFI-10325 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: nifi-nar-maven-plugin-1.3.4 >Reporter: Herve Boutemy >Priority: Major > Fix For: nifi-nar-maven-plugin-1.3.6 > > Time Spent: 50m > Remaining Estimate: 0h > > NIFI-9857 fixed most reproducibility issues > but when rebuilding NiFi 1.17.0, a remaining issue was found: the timestamp > in MANIFEST.MF uses local timezone instead of UTC as the format suggests, > then the value is different for users in different timezones > see difference #13 on .nar file in > https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/nifi/nifi-1.17.0.diffoscope -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10904) Changing Font Color for Dropdown Menu When Selecting Controller Services
Anson Yu created NIFI-10904: --- Summary: Changing Font Color for Dropdown Menu When Selecting Controller Services Key: NIFI-10904 URL: https://issues.apache.org/jira/browse/NIFI-10904 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Anson Yu Attachments: GrayOut.PNG When I was adding a controller service to a processor (such as ConvertRecords) via the Properties tab in it's configurations, I noticed the color of the base options in the dropdown containing "Reference parameter..." & "Create new service..." used a grey color that gave the impression of it being unavailable on first glance, even though were selectable options. I think it would be better if the default color was something else. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on a diff in pull request #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
greyp9 commented on code in PR #6731:
URL: https://github.com/apache/nifi/pull/6731#discussion_r1035269962
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/test/resources/Publish/parameterized/flowfileInput1.json:
##
@@ -0,0 +1,8 @@
+{
Review Comment:
some test inputs from the doc for `PublishKafka`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
greyp9 commented on code in PR #6731: URL: https://github.com/apache/nifi/pull/6731#discussion_r1035269476 ## nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/test/java/org/apache/nifi/processors/kafka/pubsub/TestPublishKafkaMockParameterized.java: ## @@ -31,17 +31,12 @@ import org.apache.kafka.clients.producer.ProducerRecord; import org.apache.kafka.clients.producer.RecordMetadata; import org.apache.kafka.common.TopicPartition; Review Comment: The previous naming was incorrect. `TestPublishKafkaMockParameterized` was testing `PublishKafkaRecord`. I copied this class to `TestPublishKafkaRecordMockParameterized`, and modified this class to test `PublishKafka`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10765) Add better error reporting for JASN1Reader
[ https://issues.apache.org/jira/browse/NIFI-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640929#comment-17640929 ] ASF subversion and git services commented on NIFI-10765: Commit f630540891dcc1a4eaa9cf2b79568ca55ef210ed in nifi's branch refs/heads/main from Tamas Palfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f630540891 ] NIFI-10765 Added better error reporting in JASN1Reader This closes #6703 Signed-off-by: David Handermann > Add better error reporting for JASN1Reader > -- > > Key: NIFI-10765 > URL: https://issues.apache.org/jira/browse/NIFI-10765 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > The JASN1Reader relies on https://github.com/beanit/asn1bean to compile ASN > schema files and to deserialize ASN data files. > The schema compilation step can fail for many different reasons and the error > is not reported to the user properly (or in any way really) so understanding > and fixing errors in the schema file is very hard currently. > We should report meaningful error messages to the bulletin and/or the > controller service validation error indicator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10765) Add better error reporting for JASN1Reader
[ https://issues.apache.org/jira/browse/NIFI-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10765. - Fix Version/s: 1.20.0 Resolution: Fixed > Add better error reporting for JASN1Reader > -- > > Key: NIFI-10765 > URL: https://issues.apache.org/jira/browse/NIFI-10765 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 1.20.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The JASN1Reader relies on https://github.com/beanit/asn1bean to compile ASN > schema files and to deserialize ASN data files. > The schema compilation step can fail for many different reasons and the error > is not reported to the user properly (or in any way really) so understanding > and fixing errors in the schema file is very hard currently. > We should report meaningful error messages to the bulletin and/or the > controller service validation error indicator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on a diff in pull request #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
greyp9 commented on code in PR #6731:
URL: https://github.com/apache/nifi/pull/6731#discussion_r103526
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublisherLease.java:
##
@@ -431,7 +431,8 @@ private void addHeaders(final FlowFile flowFile, final
Map addit
}
protected void publish(final FlowFile flowFile, final byte[] messageKey,
final byte[] messageContent, final String topic, final InFlightMessageTracker
tracker, final Integer partition) {
-publish(flowFile, Collections.emptyList(), messageKey, messageContent,
topic, tracker, partition);
Review Comment:
the defect
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #6703: NIFI-10765 - Added better error reporting in JASN1Reader.
exceptionfactory closed pull request #6703: NIFI-10765 - Added better error reporting in JASN1Reader. URL: https://github.com/apache/nifi/pull/6703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 opened a new pull request, #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
greyp9 opened a new pull request, #6731: URL: https://github.com/apache/nifi/pull/6731 ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6703: NIFI-10765 - Added better error reporting in JASN1Reader.
exceptionfactory commented on code in PR #6703:
URL: https://github.com/apache/nifi/pull/6703#discussion_r1035256497
##
nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services/src/main/java/org/apache/nifi/jasn1/JASN1Reader.java:
##
@@ -219,8 +236,55 @@ private void compileAsnToClass(String... asnFilePaths) {
asnCompilerArguments.add("-o");
asnCompilerArguments.add(asnOutDir.toString());
+HashMap modulesByName = new HashMap<>();
+
+Exception parseException = null;
+for (String asn1File : asnFilePaths) {
+logger.info("Parsing " + asn1File);
+try {
+AsnModel model = getJavaModelFromAsn1File(asn1File);
+modulesByName.putAll(model.modulesByName);
+} catch (FileNotFoundException e) {
+logger.error("Couldn't find " + asn1File, e);
+parseException = e;
+} catch (TokenStreamException | RecognitionException e) {
+logger.error("Error while parsing " + asn1File, e);
+parseException = e;
+} catch (Exception e) {
+logger.error("Couldn't parse " + asn1File, e);
+parseException = e;
+}
+}
+
+if (parseException != null) {
+throw new ProcessException("Couldn't parse asn files.",
parseException);
+}
+
try {
-
com.beanit.asn1bean.compiler.Compiler.main(asnCompilerArguments.toArray(new
String[0]));
+Constructor berClassWriterConstructor =
BerClassWriter.class.getDeclaredConstructor(
+HashMap.class,
+String.class,
+String.class,
+Boolean.TYPE,
+Boolean.TYPE,
+Boolean.TYPE
+);
+
+berClassWriterConstructor.setAccessible(true);
Review Comment:
Looks like a good solution, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-10891) TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-10891.
-
Fix Version/s: 1.20.0
Resolution: Fixed
> TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic
> HashMap
> --
>
> Key: NIFI-10891
> URL: https://issues.apache.org/jira/browse/NIFI-10891
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0
>
> Attachments: org.apache.nifi.processors.standard.TestMergeContent.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestMergeContent{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR] Failures:
> [ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028
> FlowFile content differs from input at byte 0 with input having value 65 and
> FlowFile having value 78
> {code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn clean install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
> -Drat.skip -am {noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> -Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
> The result will be saved under the module folder in .nondex
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10891) TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640925#comment-17640925
]
ASF subversion and git services commented on NIFI-10891:
Commit ee5d557d2a0a155493d0bf2b59946455a26cd792 in nifi's branch
refs/heads/main from sopan98
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ee5d557d2a ]
NIFI-10891 Replaced HashMap with LinkedHashMap in BinFiles
- Resolved non-deterministic behavior in
TestMergeContent.testDefragmentMultipleMingledSegments
This closes #6727
Signed-off-by: David Handermann
> TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic
> HashMap
> --
>
> Key: NIFI-10891
> URL: https://issues.apache.org/jira/browse/NIFI-10891
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Attachments: org.apache.nifi.processors.standard.TestMergeContent.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestMergeContent{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR] Failures:
> [ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028
> FlowFile content differs from input at byte 0 with input having value 65 and
> FlowFile having value 78
> {code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn clean install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
> -Drat.skip -am {noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> -Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
> The result will be saved under the module folder in .nondex
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-10890) TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-10890.
-
Fix Version/s: 1.20.0
Resolution: Fixed
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses
> non-deterministic HashMap
>
>
> Key: NIFI-10890
> URL: https://issues.apache.org/jira/browse/NIFI-10890
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0
>
> Attachments: org.apache.nifi.json.TestJsonTreeRowRecordReader.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.json.TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR]
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved:433
> expected: City, state=MS, zipCode=1, country=USA,
> account=MapRecord[{balance=4750.89, id=42}]}]> but was: name=John Doe, address=123 My Street, city=My City, state=MS, zipCode=1,
> country=USA, account=MapRecord[{id=42, balance=4750.89}]}]>{code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
> -DskipTests -Drat.skip -am{noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
>
> -Dtest=org.apache.nifi.json.TestJsonTreeRowRecordReader#testReadRawRecordFieldOrderPreserved
> {noformat}
> The result will be saved under the module folder in .nondex
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10890) TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640924#comment-17640924
]
ASF subversion and git services commented on NIFI-10890:
Commit ab8e9aa8df6e06fa0452c944a55e10cce39a8395 in nifi's branch
refs/heads/main from sopan98
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ab8e9aa8df ]
NIFI-10890 Replaced HashMap with LinkedHashMap in JsonRowRecordReader
- Updated unit test based on deterministic behavior of LinkedHashMap
This closes #6726
Signed-off-by: David Handermann
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses
> non-deterministic HashMap
>
>
> Key: NIFI-10890
> URL: https://issues.apache.org/jira/browse/NIFI-10890
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Attachments: org.apache.nifi.json.TestJsonTreeRowRecordReader.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.json.TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR]
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved:433
> expected: City, state=MS, zipCode=1, country=USA,
> account=MapRecord[{balance=4750.89, id=42}]}]> but was: name=John Doe, address=123 My Street, city=My City, state=MS, zipCode=1,
> country=USA, account=MapRecord[{id=42, balance=4750.89}]}]>{code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
> -DskipTests -Drat.skip -am{noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
>
> -Dtest=org.apache.nifi.json.TestJsonTreeRowRecordReader#testReadRawRecordFieldOrderPreserved
> {noformat}
> The result will be saved under the module folder in .nondex
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6726: NIFI-10890 Change HashMap to LinkedHashMap in AbstractJsonRowRecordReader.createRecordFromRawValue for deterministic handling
exceptionfactory closed pull request #6726: NIFI-10890 Change HashMap to LinkedHashMap in AbstractJsonRowRecordReader.createRecordFromRawValue for deterministic handling URL: https://github.com/apache/nifi/pull/6726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #6727: NIFI-10891 Change HashMap to LinkedHashMap in BinFiles.binFlowFiles for deterministic handling
exceptionfactory closed pull request #6727: NIFI-10891 Change HashMap to LinkedHashMap in BinFiles.binFlowFiles for deterministic handling URL: https://github.com/apache/nifi/pull/6727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10903) Fix NiFi Registry assembly profiles
Nathan Gough created NIFI-10903: --- Summary: Fix NiFi Registry assembly profiles Key: NIFI-10903 URL: https://issues.apache.org/jira/browse/NIFI-10903 Project: Apache NiFi Issue Type: Improvement Reporter: Nathan Gough Assignee: Nathan Gough NiFi Registry no longer builds the directory by default when compiling the NiFi project, and only builds a zip of the compiled assembly. Can we update this to build both zip and dir by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10902) Allow removal of specific versions from a bundle
[ https://issues.apache.org/jira/browse/NIFI-10902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] endzeit updated NIFI-10902: --- Summary: Allow removal of specific versions from a bundle (was: Allow removal of specific bundle versions) > Allow removal of specific versions from a bundle > > > Key: NIFI-10902 > URL: https://issues.apache.org/jira/browse/NIFI-10902 > Project: Apache NiFi > Issue Type: New Feature > Components: NiFi Registry >Reporter: endzeit >Priority: Major > > The NiFi registry support the upload of bundles. As of now, the only support > bundle type is "nar". > NiFi 1.16.0 added support for importing these bundles into NiFi, as outlined > in NIFI-9350. > One can remove an bundle entirely. However, removing specific (outdated) > versions of a bundle is not yet supported. > I propose a feature to remove specific versions from a bundle, in order to be > able to maintain only a selected set of supported versions. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10902) Allow removal of specific bundle versions
endzeit created NIFI-10902: -- Summary: Allow removal of specific bundle versions Key: NIFI-10902 URL: https://issues.apache.org/jira/browse/NIFI-10902 Project: Apache NiFi Issue Type: New Feature Components: NiFi Registry Reporter: endzeit The NiFi registry support the upload of bundles. As of now, the only support bundle type is "nar". NiFi 1.16.0 added support for importing these bundles into NiFi, as outlined in NIFI-9350. One can remove an bundle entirely. However, removing specific (outdated) versions of a bundle is not yet supported. I propose a feature to remove specific versions from a bundle, in order to be able to maintain only a selected set of supported versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-53) Get/Put GPIO support
[ https://issues.apache.org/jira/browse/MINIFICPP-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-53: Fix Version/s: 0.14.0 (was: 0.13.0) > Get/Put GPIO support > > > Key: MINIFICPP-53 > URL: https://issues.apache.org/jira/browse/MINIFICPP-53 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Jeremy Dyer >Priority: Major > Fix For: 0.14.0 > > > Provide processors for getting the current value of a GPIO pin or setting the > value of a GPIO pin. These functionality should be optional and will not be > built by default. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-19) Stub out properties with their defaults to minifi.properties
[ https://issues.apache.org/jira/browse/MINIFICPP-19?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-19: Fix Version/s: 0.14.0 (was: 0.13.0) > Stub out properties with their defaults to minifi.properties > > > Key: MINIFICPP-19 > URL: https://issues.apache.org/jira/browse/MINIFICPP-19 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Aldrin Piri >Assignee: Marc Parisi >Priority: Major > Labels: documentation > Fix For: 0.14.0 > > > MINIFI-37 provided some additional implementations for repositories. It > would be helpful to have these stubbed out in minifi.properites with their > defaults as well as similar other properties that a user may wish to change. > For properties not yet exposed we should do so and provide the relevant > updates. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-609) Provide framework declaration/verification of processor input requirements
[ https://issues.apache.org/jira/browse/MINIFICPP-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-609. -- Resolution: Duplicate this was done in MINIFICPP-1457 > Provide framework declaration/verification of processor input requirements > -- > > Key: MINIFICPP-609 > URL: https://issues.apache.org/jira/browse/MINIFICPP-609 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Priority: Major > > In Java variants, input requirements are specified via annotations as > outlined in the section > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#processor-behavior-annotations. > This is helpful for preventing issues with processors that will never > deplete queues and could prevent backups for instances with incorrect flows. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-520) Create .deb as part of build process
[ https://issues.apache.org/jira/browse/MINIFICPP-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-520: - Fix Version/s: 0.14.0 (was: 0.13.0) > Create .deb as part of build process > > > Key: MINIFICPP-520 > URL: https://issues.apache.org/jira/browse/MINIFICPP-520 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.14.0 > > > We already offer a debian based binary build. Adding the .deb packaging step > would significantly improve the user installation experience by allowing > users on debian based systems to quickly install on their devices. This would > also be helpful to have for quickly bootstrapping devices like Raspberry PIs > without having to physically build on them which often takes a good bit of > time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-504) Tie in estimated Size for RPG
[ https://issues.apache.org/jira/browse/MINIFICPP-504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-504: - Fix Version/s: 0.14.0 (was: 0.13.0) > Tie in estimated Size for RPG > - > > Key: MINIFICPP-504 > URL: https://issues.apache.org/jira/browse/MINIFICPP-504 > Project: Apache NiFi MiNiFi C++ > Issue Type: Sub-task >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > 457 will create the ability to select interfaces based on priority, but > throughput control isn't tied into the RPG. This needs to be wired in to > ensure we can prioritize writes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-70) More intuitively limit memory within the client.
[ https://issues.apache.org/jira/browse/MINIFICPP-70?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-70: Fix Version/s: 0.14.0 (was: 0.13.0) > More intuitively limit memory within the client. > > > Key: MINIFICPP-70 > URL: https://issues.apache.org/jira/browse/MINIFICPP-70 > Project: Apache NiFi MiNiFi C++ > Issue Type: Sub-task >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > The configuration limits memory held by connections and the back pressure > thereof, but it stands to reason that we should avoid consuming all system > memory and/or engaging the out of memory killer. We should more intuitively > allow clients to be configured such that they respect the limits of the > system. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-609) Provide framework declaration/verification of processor input requirements
[ https://issues.apache.org/jira/browse/MINIFICPP-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-609: - Fix Version/s: (was: 0.13.0) > Provide framework declaration/verification of processor input requirements > -- > > Key: MINIFICPP-609 > URL: https://issues.apache.org/jira/browse/MINIFICPP-609 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Priority: Major > > In Java variants, input requirements are specified via annotations as > outlined in the section > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#processor-behavior-annotations. > This is helpful for preventing issues with processors that will never > deplete queues and could prevent backups for instances with incorrect flows. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-612) Improve schedule/ontrigger structure of Processors
[ https://issues.apache.org/jira/browse/MINIFICPP-612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-612: - Fix Version/s: 0.14.0 (was: 0.13.0) > Improve schedule/ontrigger structure of Processors > -- > > Key: MINIFICPP-612 > URL: https://issues.apache.org/jira/browse/MINIFICPP-612 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > Processors follow Java convention in their design. This has led to following > the same control flow that exists within NiFi; however, we can take some > liberties of using template metaprogramming to improve the user experience. > This ticket will be responsible for covering onTrigger and onSchedule. The > underlying architectural decisions will remain the same, but with a focus on > allowing design time decisions to be caught and fixed at compile time versus > run time. The first area to tackle is input requirements within Processors. > Currently if we specified a limitation it wouldn't be enforced gracefully or > at all until runtime. This should not be the case. Developers who forbid > input within a processor should never have the ability to call session->get() > and this should result in a compile time error. With template metapogramming > and SFINAE, we should be able to make this a compile time restriction and > better impose enforcement within the design -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-651) Lazily start scheduling agent threads
[ https://issues.apache.org/jira/browse/MINIFICPP-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-651: - Fix Version/s: 0.14.0 (was: 0.13.0) > Lazily start scheduling agent threads > - > > Key: MINIFICPP-651 > URL: https://issues.apache.org/jira/browse/MINIFICPP-651 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Assignee: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > Right now we start these threads up. We could allow functionality that > doesn't start a thread until they are needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-660) Create docker test for python library
[ https://issues.apache.org/jira/browse/MINIFICPP-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-660: - Fix Version/s: (was: 0.13.0) > Create docker test for python library > - > > Key: MINIFICPP-660 > URL: https://issues.apache.org/jira/browse/MINIFICPP-660 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-660) Create docker test for python library
[ https://issues.apache.org/jira/browse/MINIFICPP-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-660. -- Resolution: Duplicate this was done in MINIFICPP-1355 and MINIFICPP-1695 > Create docker test for python library > - > > Key: MINIFICPP-660 > URL: https://issues.apache.org/jira/browse/MINIFICPP-660 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] malthe opened a new pull request, #6730: NIFI-10721 No querying for props after successful blob upload
malthe opened a new pull request, #6730: URL: https://github.com/apache/nifi/pull/6730 # Summary [NIFI-10721](https://issues.apache.org/jira/browse/NIFI-10721) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [x] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-775) Explore triggers for windows API calls
[ https://issues.apache.org/jira/browse/MINIFICPP-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-775: - Fix Version/s: 0.14.0 (was: 0.13.0) > Explore triggers for windows API calls > -- > > Key: MINIFICPP-775 > URL: https://issues.apache.org/jira/browse/MINIFICPP-775 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature > Environment: Windows >Reporter: Marc Parisi >Assignee: Alex Marmer >Priority: Major > Labels: Windows > Fix For: 0.14.0 > > > Explore creating triggers on a windows system that allow us to capture events > and create provenance when needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-755) Link provenance reporting
[ https://issues.apache.org/jira/browse/MINIFICPP-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-755: - Fix Version/s: 0.14.0 (was: 0.13.0) > Link provenance reporting > - > > Key: MINIFICPP-755 > URL: https://issues.apache.org/jira/browse/MINIFICPP-755 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Priority: Major > Labels: provenance > Fix For: 0.14.0 > > > Ticket should also cover making ProvenanceReportingTest executed as part of > ctest and add meaningful verifications into that testcase. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-721) Add time in queue/repo
[ https://issues.apache.org/jira/browse/MINIFICPP-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-721: - Fix Version/s: 0.14.0 (was: 0.13.0) > Add time in queue/repo > -- > > Key: MINIFICPP-721 > URL: https://issues.apache.org/jira/browse/MINIFICPP-721 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Assignee: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > Add metrics for time in queue to better debug various parameters. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-778) Docker build and test directories should be copied into build directory, not source tree
[ https://issues.apache.org/jira/browse/MINIFICPP-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-778. -- Resolution: Resolved I'm not sure what the issue was, but I'm fairly sure it has been resolved since then. > Docker build and test directories should be copied into build directory, not > source tree > > > Key: MINIFICPP-778 > URL: https://issues.apache.org/jira/browse/MINIFICPP-778 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-778) Docker build and test directories should be copied into build directory, not source tree
[ https://issues.apache.org/jira/browse/MINIFICPP-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-778: - Fix Version/s: (was: 0.13.0) > Docker build and test directories should be copied into build directory, not > source tree > > > Key: MINIFICPP-778 > URL: https://issues.apache.org/jira/browse/MINIFICPP-778 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-815) Add more FileUtils tests
[ https://issues.apache.org/jira/browse/MINIFICPP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-815: - Fix Version/s: 0.14.0 (was: 0.13.0) > Add more FileUtils tests > > > Key: MINIFICPP-815 > URL: https://issues.apache.org/jira/browse/MINIFICPP-815 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > > Formerly titled: > [Dog fooded test should use a separate test function to validate the lamdba > function|https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > [ > https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > is a dog fooded test. > > We should probably make a FileUtils suite of tests as a focal point than > specifically addressing the above class's tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-782) Better define deprecated macros
[ https://issues.apache.org/jira/browse/MINIFICPP-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-782: - Fix Version/s: 0.14.0 (was: 0.13.0) > Better define deprecated macros > --- > > Key: MINIFICPP-782 > URL: https://issues.apache.org/jira/browse/MINIFICPP-782 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-815) Add more FileUtils tests
[ https://issues.apache.org/jira/browse/MINIFICPP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-815: - Summary: Add more FileUtils tests (was: Add FileUtils Tests) > Add more FileUtils tests > > > Key: MINIFICPP-815 > URL: https://issues.apache.org/jira/browse/MINIFICPP-815 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > Fix For: 0.13.0 > > > Formerly titled: > [Dog fooded test should use a separate test function to validate the lamdba > function|https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > [ > https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > is a dog fooded test. > > We should probably make a FileUtils suite of tests as a focal point than > specifically addressing the above class's tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-821) Add Metrics to all processors
[ https://issues.apache.org/jira/browse/MINIFICPP-821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-821. -- Resolution: Duplicate duplicate of MINIFICPP-1848 > Add Metrics to all processors > - > > Key: MINIFICPP-821 > URL: https://issues.apache.org/jira/browse/MINIFICPP-821 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Priority: Major > > GetFile and GetTCP Metrics are examples where we can gather processor time > metrics to inject into command and control responses. We should add these to > processors where it makes sense. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-824) MacOS build uses RocksDB from brew, if installed, causing segfaults
[ https://issues.apache.org/jira/browse/MINIFICPP-824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-824: - Fix Version/s: 0.14.0 (was: 0.13.0) > MacOS build uses RocksDB from brew, if installed, causing segfaults > --- > > Key: MINIFICPP-824 > URL: https://issues.apache.org/jira/browse/MINIFICPP-824 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Dániel Bakai >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > > If RocksDB is installed from brew, libminifi/test/rocksdb-tests/RepoTests > segfaults, because of a wrong call to rocksdb::DBImpl's destructor (instead > of the destructor, an another member function is called). > It seems like somehow the shipped and the system versions mix up, probably > causing an ODR violation and ultimately the segfault. (Or the version shipped > in brew is _very_ incompatible.) > After RocksDB was uninstalled from brew, and minifi was rebuilt, the test ran > correctly. > This might affect other platforms as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-832) Add content route into ProcessSession
[ https://issues.apache.org/jira/browse/MINIFICPP-832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-832: - Fix Version/s: 0.14.0 (was: 0.13.0) > Add content route into ProcessSession > - > > Key: MINIFICPP-832 > URL: https://issues.apache.org/jira/browse/MINIFICPP-832 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > In lieu of MINIFICPP-727 we can add routing capabilities to process session. > We can support the same general concepts, and do it much earlier. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-821) Add Metrics to all processors
[ https://issues.apache.org/jira/browse/MINIFICPP-821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-821: - Fix Version/s: (was: 0.13.0) > Add Metrics to all processors > - > > Key: MINIFICPP-821 > URL: https://issues.apache.org/jira/browse/MINIFICPP-821 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Priority: Major > > GetFile and GetTCP Metrics are examples where we can gather processor time > metrics to inject into command and control responses. We should add these to > processors where it makes sense. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-833) Integration of python bootstrap
[ https://issues.apache.org/jira/browse/MINIFICPP-833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-833: - Fix Version/s: 0.14.0 (was: 0.13.0) > Integration of python bootstrap > --- > > Key: MINIFICPP-833 > URL: https://issues.apache.org/jira/browse/MINIFICPP-833 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > > Mirror selection, release and binary lookup was implemented in scope of > MINIFICPP-672 although it's not yet integrated to the current bootstrap > functionality. > The scope of this ticket is to decide whether to leave the Python script as a > standalone (needs some changes in that case) or integrate into the current > bootstrap shell script? > Questions here: > -What about Win? > -Can we assume all targets having Python 2.7 at least or should be > conditionally included? > Some documentation would be useful, too. > [~phrocker] what's your opinion on this? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-834) Create static links for lzma
[ https://issues.apache.org/jira/browse/MINIFICPP-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-834: - Fix Version/s: 0.14.0 (was: 0.13.0) > Create static links for lzma > > > Key: MINIFICPP-834 > URL: https://issues.apache.org/jira/browse/MINIFICPP-834 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > Original Estimate: 72h > Remaining Estimate: 72h > > Statically link lzma when shared libs are turned off. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-836) Validate stop and notifystop allow processors to flush
[ https://issues.apache.org/jira/browse/MINIFICPP-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-836: - Fix Version/s: 0.14.0 (was: 0.13.0) > Validate stop and notifystop allow processors to flush > -- > > Key: MINIFICPP-836 > URL: https://issues.apache.org/jira/browse/MINIFICPP-836 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Marc Parisi >Assignee: Marc Parisi >Priority: Blocker > Labels: ReloadValidation > Fix For: 0.14.0 > > > In the event that a flow update occurs,validate that notify stop allows all > processors to complete and flush. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-889) Create MiNiFI NAR equivalent
[ https://issues.apache.org/jira/browse/MINIFICPP-889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-889: - Fix Version/s: 0.14.0 (was: 0.13.0) > Create MiNiFI NAR equivalent > - > > Key: MINIFICPP-889 > URL: https://issues.apache.org/jira/browse/MINIFICPP-889 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > > Create MiNiFI NAR equivalent so that we can integrate with Apache NiFi > registry. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-859) Prioritize connection queues.
[ https://issues.apache.org/jira/browse/MINIFICPP-859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-859: - Fix Version/s: 0.14.0 (was: 0.13.0) > Prioritize connection queues. > -- > > Key: MINIFICPP-859 > URL: https://issues.apache.org/jira/browse/MINIFICPP-859 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Priority: Major > Fix For: 0.14.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-899) Add sensitive property capabilities.
[ https://issues.apache.org/jira/browse/MINIFICPP-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-899. -- Resolution: Duplicate duplicate of MINIFICPP-1323 > Add sensitive property capabilities. > - > > Key: MINIFICPP-899 > URL: https://issues.apache.org/jira/browse/MINIFICPP-899 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Assignee: Marc Parisi >Priority: Major > Labels: c2 > > Should be able to add sensitive markers for properties. This matters more for > the manifest that is produced via c2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-912) Add GetSFTP Support
[ https://issues.apache.org/jira/browse/MINIFICPP-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-912: - Fix Version/s: 0.14.0 (was: 0.13.0) > Add GetSFTP Support > --- > > Key: MINIFICPP-912 > URL: https://issues.apache.org/jira/browse/MINIFICPP-912 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Dániel Bakai >Priority: Major > Fix For: 0.14.0 > > > SFTP is an easy operation to add through the use of libCURL > > Processors the can be added: > GetSFTP -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-899) Add sensitive property capabilities.
[ https://issues.apache.org/jira/browse/MINIFICPP-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-899: - Fix Version/s: (was: 0.13.0) > Add sensitive property capabilities. > - > > Key: MINIFICPP-899 > URL: https://issues.apache.org/jira/browse/MINIFICPP-899 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marc Parisi >Assignee: Marc Parisi >Priority: Major > Labels: c2 > > Should be able to add sensitive markers for properties. This matters more for > the manifest that is produced via c2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-981) Avoid maintaining our own Curl HTTP Client
[ https://issues.apache.org/jira/browse/MINIFICPP-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-981: - Fix Version/s: 0.14.0 (was: 0.13.0) > Avoid maintaining our own Curl HTTP Client > -- > > Key: MINIFICPP-981 > URL: https://issues.apache.org/jira/browse/MINIFICPP-981 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marc Parisi >Priority: Critical > Fix For: 0.14.0 > > > [https://github.com/jpbarrette/curlpp] may be a better alternative. I use it > on a separate project and would have preferred starting with that. Some > stakeholders did not want us to use that library, but I think we should avoid > maintaining our own client and wasting cycles to fix bugs. While we can argue > that we "know what our bugs are" I don't think that we do and there are > likely many more. Through evolution curlpp has improved and I would not like > to incur that cost on our project. I would prefer to avoid maintaining curl > code and use a third party library. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-967) Improve SITs
[ https://issues.apache.org/jira/browse/MINIFICPP-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-967: - Fix Version/s: 0.14.0 (was: 0.13.0) > Improve SITs > > > Key: MINIFICPP-967 > URL: https://issues.apache.org/jira/browse/MINIFICPP-967 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Marc Parisi >Assignee: Arpad Boda >Priority: Blocker > Fix For: 0.14.0 > > > I'm using this ticket to collect various other tickets ( closed and open ) > for adding them to our docker test framework. > > In my opinion this is the highest need in 0.7.0 in 0.8.0 and 0.9.0. Bugs will > arise as a result of this; however, there have been a variety of tickets > created and closed across this and elsewhere – so let's gather them into a > single EPIC so we can all work on them together. > > docker-verify > We have several stages of testing as you have across any project: unit, > integration, and SIT ( system integration testing ). > > 1) V – Verify and validate interactions between internal and external > components > In many cases, we can create all appropriate BV, EC, and introspective > test cases in unit and integration tests, but there are many that we cannot > capture without integration with a C2 server, NiFi Instance, Kafka server, or > other type of server. We should not attempt to launch these services in unit > or integration tests. External services and requirements for comms should be > verified and validated in docker contianers. > 2) Regression Testing – we should use this for all releases and before > merging. This may mean adding a targets if necessary, but the important thing > is that we define expectations within this test framework ( and the DSL ) and > adhere to them. This may mean that we are testing for memory leaks. We can do > this in docker tests more easily than unit/integration tests. ASAN is a great > goal to have for upcoming releases and I think docker tests are where we can > build this functionality into so that we can test across versions of systems, > glibc, and musl. > > 3) We have many versions of software that we can build upon. While we can > mitigate factors with including our own version of system libs, this won't > solve all issues, and will not be something that all consumers want. So we > can mitigate these desires by using docker tests to splay across platforms > and build systems, executing docker tests. > > 4) Fuzzing. Fuzzing can and often should be done at every level, but we can > very easily define ways in the DSL to fuzz properties in processors. This > mechanism should allow us to better test boundary values and equivalence > classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1019) Python Processor Building, Setup, and Configuration
[ https://issues.apache.org/jira/browse/MINIFICPP-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1019: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Python Processor Building, Setup, and Configuration > --- > > Key: MINIFICPP-1019 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1019 > Project: Apache NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.14.0 > > > Create documentation for building the minifi binary for Python processor > support, creating Python processors, and configuring the minifi agent runtime > to run your created processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-990) Segfault on SIGINT after shutdown initiated
[ https://issues.apache.org/jira/browse/MINIFICPP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-990: - Fix Version/s: 0.14.0 (was: 0.13.0) > Segfault on SIGINT after shutdown initiated > > > Key: MINIFICPP-990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Priority: Major > Fix For: 0.14.0 > > > If I send a SIGINT (ctrl-c) to MiNiFI, MiNiFi begins shutdown. If I send > another SIGINT while that shutdown is in progress, MiNiFi crashes with a > segfault. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1053) An error is logged when loading a YAML configuration with 'type' instead of 'class'
[ https://issues.apache.org/jira/browse/MINIFICPP-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1053: -- Fix Version/s: 0.14.0 (was: 0.13.0) > An error is logged when loading a YAML configuration with 'type' instead of > 'class' > --- > > Key: MINIFICPP-1053 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1053 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Dániel Bakai >Assignee: Nghia Le >Priority: Major > Fix For: 0.14.0 > > > In YamlConfiguration::parseControllerServices we try to first find `class` > and fall back to `type` if it does not exist, but we log a scary error during > the process, which might confuse users, since despite it, we are able to > successfully parse the configuration: > `[2019-10-04 16:23:48.093] > [org::apache::nifi::minifi::core::YamlConfiguration] [error] Unable to parse > configuration file for component named 'SSLContextService' as required field > 'class' is missing [in 'Controller Services' section of configuration file]` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1100) Replace custom base 64 encoding with Boost or alternative.
[ https://issues.apache.org/jira/browse/MINIFICPP-1100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1100: -- Fix Version/s: (was: 0.13.0) > Replace custom base 64 encoding with Boost or alternative. > --- > > Key: MINIFICPP-1100 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1100 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marc Parisi >Priority: Blocker > > Per the discussion of MINIFICPP-1026, I think using and referencing boost lib > for not only this component but also others is ideal. > "Yep. I was 1. My gut suggestion was to use boost's base 64 since it's > relatively isolated. If you look at the boost variant they mention portions > come from this repo ( [https://github.com/ReneNyffenegger/cpp-base64])" > > Since we have increased the build with static libs we might as well begin > explore simply using boost directly. The size of the build is moot at that > point. We should have rationale why custom libs are added to our core when > alternatives exist that are more widely used. > Build time, if we only reference portions of boost, should not increase > dramatically. > > I believe that this should occur despite the inertia of the expedient merge > and should function as a augmentation of the tests and replacement of the > custom code in StringUtils. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1075) Heartbeats over CoAP are incomplete when running with empty flow
[ https://issues.apache.org/jira/browse/MINIFICPP-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1075: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Heartbeats over CoAP are incomplete when running with empty flow > > > Key: MINIFICPP-1075 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1075 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Aldrin Piri >Assignee: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > > In the event that the CoAP protocol is used but the instance has an empty > config.yml there will be an ignorable PayloadParseException about queues not > being available but this precludes the rest of the hearbeat from being formed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1055) Investigate preprocessor expressions including __GNUC__
[ https://issues.apache.org/jira/browse/MINIFICPP-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1055: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Investigate preprocessor expressions including __GNUC__ > --- > > Key: MINIFICPP-1055 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1055 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Dániel Bakai >Assignee: Dániel Bakai >Priority: Major > Fix For: 0.14.0 > > > These can disable functionality if we use clang (Linux alternative or macOS) > or MSVC. > > Our problematic __GNUC__ usages fall into the following categories: > Warning disable pragmas: > nanofi/include/sitetosite/CSiteToSite.h > extensions/standard-processors/processors/TailFile.cpp > extensions/standard-processors/processors/ExecuteProcess.cpp > libminifi/test/unit/ProvenanceTestHelper.h > libminifi/include/core/repository/VolatileRepository.h > libminifi/include/sitetosite/SiteToSite.h > It may be beneficial to create common warning disabling macros instead of > these individual ones: https://issues.apache.org/jira/browse/MINIFICPP-1211 > Other usages: > libminifi/include/core/yaml/YamlConfiguration.h -> Regex usage > (https://issues.apache.org/jira/browse/MINIFICPP-1212) > extensions/expression-language/impl/expression/Expression.h -> Regex and > date usage (https://issues.apache.org/jira/browse/MINIFICPP-1054) > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10785) Allow publishing AMQP message with null header value
[ https://issues.apache.org/jira/browse/NIFI-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-10785: Fix Version/s: 1.20.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Allow publishing AMQP message with null header value > > > Key: NIFI-10785 > URL: https://issues.apache.org/jira/browse/NIFI-10785 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: amqp > Fix For: 1.20.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Since after NIFI-10317 ConsumeAMQP is able to handle null header value it > makes sense to support it in PublishAMQP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1138) moodycamel::concurrentqueue never relinquishes memory
[ https://issues.apache.org/jira/browse/MINIFICPP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1138: -- Fix Version/s: 0.14.0 (was: 0.13.0) > moodycamel::concurrentqueue never relinquishes memory > - > > Key: MINIFICPP-1138 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1138 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Dániel Bakai >Priority: Blocker > Fix For: 0.14.0 > > > This is especially problematic with CWEL, but we use it in many places. > Once memory is allocated via enqueue, it will never be freed until the entire > queue is deallocated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1143) nifi.c2.rest.ssl.context.service property is broken
[ https://issues.apache.org/jira/browse/MINIFICPP-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1143: -- Fix Version/s: 0.14.0 (was: 0.13.0) > nifi.c2.rest.ssl.context.service property is broken > --- > > Key: MINIFICPP-1143 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1143 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Dániel Bakai >Priority: Blocker > Fix For: 0.14.0 > > > C2 over SSL can only be configured via the nifi.security.client.* properties, > if an SSLContextService is supplied via nifi.c2.rest.ssl.context.service, > setSecurityContext does not get called and the connection will fail. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1145) Create a common RocksdbRepository baseclass
[ https://issues.apache.org/jira/browse/MINIFICPP-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1145: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Create a common RocksdbRepository baseclass > --- > > Key: MINIFICPP-1145 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1145 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.14.0 > > > As MINIFICPP-1126 and MINIFICPP-1127 highlighted there is a lot in common in > Provenance and FlowFile repositories. > Rocksdb-based content repo and the planned rocksdb-based processor state > storage would also result in some code duplication. > printStats should be moved to the baseclass and properly handle failed getter > calls. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1156) Avoid recursive mutex in Soci SQL impl
[ https://issues.apache.org/jira/browse/MINIFICPP-1156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1156: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Avoid recursive mutex in Soci SQL impl > -- > > Key: MINIFICPP-1156 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1156 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Priority: Major > Fix For: 0.14.0 > > > Recursive mutex is usually a bad smell and should be avoided. > A possible workaround is unique locks passed as arguments. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1178) Make the state storage queryable and per-component clearable through C2
[ https://issues.apache.org/jira/browse/MINIFICPP-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1178: -- Fix Version/s: (was: 0.13.0) > Make the state storage queryable and per-component clearable through C2 > --- > > Key: MINIFICPP-1178 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1178 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Dániel Bakai >Priority: Major > > With the new state storage it is not trivial anymore to delete the state of a > single processor. > DESCRIBE corecomponentstate has been added and tested. > CLEAR corecomponentstate had been added, but it is untested. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10785) Allow publishing AMQP message with null header value
[ https://issues.apache.org/jira/browse/NIFI-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640868#comment-17640868 ] ASF subversion and git services commented on NIFI-10785: Commit 3a7ec5d5429606eb6e15592aedc97434f080fd13 in nifi's branch refs/heads/main from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3a7ec5d542 ] NIFI-10785 Allow publishing AMQP message with null header value NIFI-10785 addressing review comment NIFI-10785 addressing review comments (remove unnecessary property to ignore null headers) Signed-off-by: Nathan Gough This closes #6649. > Allow publishing AMQP message with null header value > > > Key: NIFI-10785 > URL: https://issues.apache.org/jira/browse/NIFI-10785 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: amqp > Time Spent: 1h 50m > Remaining Estimate: 0h > > Since after NIFI-10317 ConsumeAMQP is able to handle null header value it > makes sense to support it in PublishAMQP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1258) Remove anonymous namespace from HashContent.h
[ https://issues.apache.org/jira/browse/MINIFICPP-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1258: -- Fix Version/s: (was: 0.13.0) > Remove anonymous namespace from HashContent.h > - > > Key: MINIFICPP-1258 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1258 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Priority: Trivial > Labels: MiNiFi-CPP-Hygiene > > *Background:* > HashContent.h contains an anonymous namespace in order to resolve linktime > symbol collisions. > *Proposal:* > One should investigate the underlying reason and potentially separate the > implementation from the declaration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog closed pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value
thenatog closed pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value URL: https://github.com/apache/nifi/pull/6649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1288) Implement static flow configuration validation
[ https://issues.apache.org/jira/browse/MINIFICPP-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-1288. --- Resolution: Abandoned > Implement static flow configuration validation > -- > > Key: MINIFICPP-1288 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1288 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Priority: Major > Labels: selected-for-development > Time Spent: 7.5h > Remaining Estimate: 0h > > *Background:* > MiNiFi flows are represented by yaml configuration files. When these are > published to the FlowController in MiNiFi, it handles this as a “reconfigure > and restart”. The current logic is: > # Backup the old process group (in memory representation of the > configuration). > # Stop the FlowController. > # Try to build up a new process group based on the new configuration. > # If we fail during the construction of the process group nodes we should > keep the backup, otherwise we let the new flow run and consider it validated. > This is not awful, but expects the checks implemented in terms of processor > construction-time checks and separated cheks on the connection network. [[The > implementation is also not very > robust]|https://github.com/apache/nifi-minifi-cpp/blob/rel/minifi-cpp-0.7.0/libminifi/src/core/ProcessGroup.cpp#L343-L359], > so there is not much extra value in keeping it. > *Proposal:* > We should try and validate the configuration files before trying to reload > using them. Not only would this make us fail earlier, but would have the > checks a bit more decoupled from the processors. We already send processor > configuratin requirement data via the heartbeat to EFM, so we have every > piece of information needed to statically validate the configuration files. > This way the new logic should simplify to: > # Validate the configuration > # If it is valid load it, otherwise load it and any subsequent error should > be considered runtime exception. > (!) As this would potentially break configuration-file backward > compatibilty, all the changes should be implemented behind a feature flag > (build-time). > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1325) Rework usages of UUID and name for identifying components
[ https://issues.apache.org/jira/browse/MINIFICPP-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-1325: - Assignee: (was: Adam Hunyadi) > Rework usages of UUID and name for identifying components > - > > Key: MINIFICPP-1325 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1325 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Time Spent: 8h 10m > Remaining Estimate: 0h > > *Background:* > We are using both names and UUID-s to identify core components. The > differentation between responsibilities between these are not clear and this > results in cases where we do lookups on both the lists of UUID-s and the > names "just in case". > *Proposal:* > We should enforce uniqueness (eg. prohibiting users from setting it manually) > of the UUID and make sure that each of the CoreComponents have it set. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1288) Implement static flow configuration validation
[ https://issues.apache.org/jira/browse/MINIFICPP-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1288: -- Fix Version/s: (was: 0.13.0) > Implement static flow configuration validation > -- > > Key: MINIFICPP-1288 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1288 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Assignee: Adam Hunyadi >Priority: Major > Labels: selected-for-development > Time Spent: 7.5h > Remaining Estimate: 0h > > *Background:* > MiNiFi flows are represented by yaml configuration files. When these are > published to the FlowController in MiNiFi, it handles this as a “reconfigure > and restart”. The current logic is: > # Backup the old process group (in memory representation of the > configuration). > # Stop the FlowController. > # Try to build up a new process group based on the new configuration. > # If we fail during the construction of the process group nodes we should > keep the backup, otherwise we let the new flow run and consider it validated. > This is not awful, but expects the checks implemented in terms of processor > construction-time checks and separated cheks on the connection network. [[The > implementation is also not very > robust]|https://github.com/apache/nifi-minifi-cpp/blob/rel/minifi-cpp-0.7.0/libminifi/src/core/ProcessGroup.cpp#L343-L359], > so there is not much extra value in keeping it. > *Proposal:* > We should try and validate the configuration files before trying to reload > using them. Not only would this make us fail earlier, but would have the > checks a bit more decoupled from the processors. We already send processor > configuratin requirement data via the heartbeat to EFM, so we have every > piece of information needed to statically validate the configuration files. > This way the new logic should simplify to: > # Validate the configuration > # If it is valid load it, otherwise load it and any subsequent error should > be considered runtime exception. > (!) As this would potentially break configuration-file backward > compatibilty, all the changes should be implemented behind a feature flag > (build-time). > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1288) Implement static flow configuration validation
[ https://issues.apache.org/jira/browse/MINIFICPP-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-1288: - Assignee: (was: Adam Hunyadi) > Implement static flow configuration validation > -- > > Key: MINIFICPP-1288 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1288 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Priority: Major > Labels: selected-for-development > Time Spent: 7.5h > Remaining Estimate: 0h > > *Background:* > MiNiFi flows are represented by yaml configuration files. When these are > published to the FlowController in MiNiFi, it handles this as a “reconfigure > and restart”. The current logic is: > # Backup the old process group (in memory representation of the > configuration). > # Stop the FlowController. > # Try to build up a new process group based on the new configuration. > # If we fail during the construction of the process group nodes we should > keep the backup, otherwise we let the new flow run and consider it validated. > This is not awful, but expects the checks implemented in terms of processor > construction-time checks and separated cheks on the connection network. [[The > implementation is also not very > robust]|https://github.com/apache/nifi-minifi-cpp/blob/rel/minifi-cpp-0.7.0/libminifi/src/core/ProcessGroup.cpp#L343-L359], > so there is not much extra value in keeping it. > *Proposal:* > We should try and validate the configuration files before trying to reload > using them. Not only would this make us fail earlier, but would have the > checks a bit more decoupled from the processors. We already send processor > configuratin requirement data via the heartbeat to EFM, so we have every > piece of information needed to statically validate the configuration files. > This way the new logic should simplify to: > # Validate the configuration > # If it is valid load it, otherwise load it and any subsequent error should > be considered runtime exception. > (!) As this would potentially break configuration-file backward > compatibilty, all the changes should be implemented behind a feature flag > (build-time). > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1730) Ship systemd service file
[ https://issues.apache.org/jira/browse/MINIFICPP-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1730: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Ship systemd service file > - > > Key: MINIFICPP-1730 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1730 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1372) Add batch cleanup for ContentRepository
[ https://issues.apache.org/jira/browse/MINIFICPP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1372: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Add batch cleanup for ContentRepository > --- > > Key: MINIFICPP-1372 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1372 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.9.0, 0.7.0 >Reporter: Adam Debreceni >Priority: Major > Labels: selected-for-development > Fix For: 0.14.0 > > > Currently all content is removed one-by-one whenever the refCount in the > ContentReposiroty::count_map_ goes to 0. For the rocksdb backed > DatabaseContentRepository this results in an unnecessary cost we could elide > if we batch-remove these resources. We could create a background gc thread > like in FlowFileRepository to handle the actual removal. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1325) Rework usages of UUID and name for identifying components
[ https://issues.apache.org/jira/browse/MINIFICPP-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1325: -- Fix Version/s: (was: 0.13.0) > Rework usages of UUID and name for identifying components > - > > Key: MINIFICPP-1325 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1325 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Adam Hunyadi >Assignee: Adam Hunyadi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Time Spent: 8h 10m > Remaining Estimate: 0h > > *Background:* > We are using both names and UUID-s to identify core components. The > differentation between responsibilities between these are not clear and this > results in cases where we do lookups on both the lists of UUID-s and the > names "just in case". > *Proposal:* > We should enforce uniqueness (eg. prohibiting users from setting it manually) > of the UUID and make sure that each of the CoreComponents have it set. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1862) Change leftover boost filesystem usages from MINIFICPP-1636
[ https://issues.apache.org/jira/browse/MINIFICPP-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1862: -- Fix Version/s: 0.14.0 (was: 0.13.0) > Change leftover boost filesystem usages from MINIFICPP-1636 > --- > > Key: MINIFICPP-1862 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1862 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > There are a few leftover boost::filesystem usages which could be removed. > We could try to move all usage of std::string to std::filesystem:path where > the variable represents a path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
