[jira] [Updated] (NIFI-13824) Install Python processor dependencies in one go
[ https://issues.apache.org/jira/browse/NIFI-13824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated NIFI-13824: -- Description: Dependencies for Python processors are defined in two places: the {{requirements.txt}} file in the package directory and the {{ProcessorDetails.dependencies}} field in the processor class. Currently, NiFi first installs the dependencies defined in the former, and then separately those defined in the latter. This is a problem, because a dependency in the second batch may depend on e.g., {{somepackage<2.0}} and if the first batch has already installed version {{2.3}} of {{{}somepackage{}}}, then the installation will fail. In almost all cases, {{pip}} can find dependency versions which satisfy both sets of dependencies, but only if they are all listed in a single {{pip install}} command. So we need to merge the two sets of dependencies and install them in one go. was: Dependencies for Python processors are defined in two places: the {{requirements.txt}} file in the package directory and the {{ProcessorDetails.dependencies}} field in the processor class. Currently, NiFi first installs the dependencies defined in the former, and then separately those defined in the latter. This is a problem, because a dependency in the second batch may depend on e.g., {{somepackage<2.0}} and if the first batch has already installed version {{2.3}} of {{somepackage}}, then the installation will fail. In almost all cases, {{pip}} can find dependency versions which satisfy both sets of dependencies, but only if they are all listed in a single {{pip install}} command. So we need to merge the two sets dependencies and install them in one go. > Install Python processor dependencies in one go > --- > > Key: NIFI-13824 > URL: https://issues.apache.org/jira/browse/NIFI-13824 > Project: Apache NiFi > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > Dependencies for Python processors are defined in two places: the > {{requirements.txt}} file in the package directory and the > {{ProcessorDetails.dependencies}} field in the processor class. Currently, > NiFi first installs the dependencies defined in the former, and then > separately those defined in the latter. > This is a problem, because a dependency in the second batch may depend on > e.g., {{somepackage<2.0}} and if the first batch has already installed > version {{2.3}} of {{{}somepackage{}}}, then the installation will fail. > In almost all cases, {{pip}} can find dependency versions which satisfy both > sets of dependencies, but only if they are all listed in a single {{pip > install}} command. So we need to merge the two sets of dependencies and > install them in one go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13824) Install Python processor dependencies in one go
Ferenc Gerlits created NIFI-13824: - Summary: Install Python processor dependencies in one go Key: NIFI-13824 URL: https://issues.apache.org/jira/browse/NIFI-13824 Project: Apache NiFi Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Dependencies for Python processors are defined in two places: the {{requirements.txt}} file in the package directory and the {{ProcessorDetails.dependencies}} field in the processor class. Currently, NiFi first installs the dependencies defined in the former, and then separately those defined in the latter. This is a problem, because a dependency in the second batch may depend on e.g., {{somepackage<2.0}} and if the first batch has already installed version {{2.3}} of {{somepackage}}, then the installation will fail. In almost all cases, {{pip}} can find dependency versions which satisfy both sets of dependencies, but only if they are all listed in a single {{pip install}} command. So we need to merge the two sets dependencies and install them in one go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2457) C2AssetSyncTest fails transiently in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2457: -- Epic Link: MINIFICPP-1324 > C2AssetSyncTest fails transiently in CI > --- > > Key: MINIFICPP-2457 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2457 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Priority: Minor > Attachments: C2AssetSyncTest_failure.log > > > [https://github.com/apache/nifi-minifi-cpp/actions/runs/10956698530/attempts/1] > attached logs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2457) C2AssetSyncTest fails transiently in CI
Ferenc Gerlits created MINIFICPP-2457: - Summary: C2AssetSyncTest fails transiently in CI Key: MINIFICPP-2457 URL: https://issues.apache.org/jira/browse/MINIFICPP-2457 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Attachments: C2AssetSyncTest_failure.log [https://github.com/apache/nifi-minifi-cpp/actions/runs/10956698530/attempts/1] attached logs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2451) Upgrade aws-sdk-cpp to fix a build error on msvc 19.41
[ https://issues.apache.org/jira/browse/MINIFICPP-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2451. --- Resolution: Fixed > Upgrade aws-sdk-cpp to fix a build error on msvc 19.41 > -- > > Key: MINIFICPP-2451 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2451 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Since the upgrade of the Github Action Windows runners from msvc 19.40 to > 19.41, the aws-sdk-cpp dependency no longer compiles on Windows. > This is due to some refactoring of headers in the msvc standard library, and > the fact that the StringView.h did not explicitly include the {{}} > header. It must have been included indirectly, but it no longer is. > Upgrade the aws-sdk-cpp dependency to a version which fixes the build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2422) Remove ubuntu 20.04 support
[ https://issues.apache.org/jira/browse/MINIFICPP-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2422: - Assignee: (was: Ferenc Gerlits) > Remove ubuntu 20.04 support > --- > > Key: MINIFICPP-2422 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2422 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Ferenc Gerlits >Priority: Minor > > Remove support for ubuntu 20.04 everywhere, replace with either ubuntu 22.04 > or 24.04. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2451) Upgrade aws-sdk-cpp to fix a build error on msvc 19.41
Ferenc Gerlits created MINIFICPP-2451: - Summary: Upgrade aws-sdk-cpp to fix a build error on msvc 19.41 Key: MINIFICPP-2451 URL: https://issues.apache.org/jira/browse/MINIFICPP-2451 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Since the upgrade of the Github Action Windows runners from msvc 19.40 to 19.41, the aws-sdk-cpp dependency no longer compiles on Windows. This is due to some refactoring of headers in the msvc standard library, and the fact that the StringView.h did not explicitly include the {{}} header. It must have been included indirectly, but it no longer is. Upgrade the aws-sdk-cpp dependency to a version which fixes the build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2447) Include specific Python processor version in the manifest (if available)
[ https://issues.apache.org/jira/browse/MINIFICPP-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2447: -- Description: NiFi python processors have a {{ProcessorDetails.version}} field, but this is ignored when MiNiFi creates the manifest. Instead, the MiNiFi binary's version number is shown as the processor version in the manifest. When {{ProcessorDetails.version}} is available, we should use this as the processor version in the manifest, and only fall back to the MiNiFi binary's version when it's not available. was: NiFi python processors have a {{ProcessorDetails.version}} field, but this is ignored when creating the manifest, and the processor gets the MiNiFi binary's version number as the processor version. When {{ProcessorDetails.version}} is available, we should use this as the processor version in the manifest, and only fall back to the MiNiFi binary's version when it's not available. > Include specific Python processor version in the manifest (if available) > > > Key: MINIFICPP-2447 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2447 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > NiFi python processors have a {{ProcessorDetails.version}} field, but this is > ignored when MiNiFi creates the manifest. Instead, the MiNiFi binary's > version number is shown as the processor version in the manifest. > When {{ProcessorDetails.version}} is available, we should use this as the > processor version in the manifest, and only fall back to the MiNiFi binary's > version when it's not available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2447) Include specific Python processor version in the manifest (if available)
Ferenc Gerlits created MINIFICPP-2447: - Summary: Include specific Python processor version in the manifest (if available) Key: MINIFICPP-2447 URL: https://issues.apache.org/jira/browse/MINIFICPP-2447 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits NiFi python processors have a {{ProcessorDetails.version}} field, but this is ignored when creating the manifest, and the processor gets the MiNiFi binary's version number as the processor version. When {{ProcessorDetails.version}} is available, we should use this as the processor version in the manifest, and only fall back to the MiNiFi binary's version when it's not available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2441) Use to_array when creating Properties
[ https://issues.apache.org/jira/browse/MINIFICPP-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2441: - Assignee: James Guzman (Medel) > Use to_array when creating Properties > - > > Key: MINIFICPP-2441 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2441 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: James Guzman (Medel) >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When creating the static Properties member of Processor and ControllerService > objects, we currently use the `std::array` template constructor directly to > convert the PropertyDefinition objects to PropertyReference objects, which > means we have to manually specify the number of properties. This works, but > it isn't elegant. We could use the `std::to_array()` function to do the > conversion instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2441) Use to_array when creating Properties
[ https://issues.apache.org/jira/browse/MINIFICPP-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2441: -- Fix Version/s: 0.99.1 > Use to_array when creating Properties > - > > Key: MINIFICPP-2441 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2441 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: James Guzman (Medel) >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 10m > Remaining Estimate: 0h > > When creating the static Properties member of Processor and ControllerService > objects, we currently use the `std::array` template constructor directly to > convert the PropertyDefinition objects to PropertyReference objects, which > means we have to manually specify the number of properties. This works, but > it isn't elegant. We could use the `std::to_array()` function to do the > conversion instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2437) Include allowable_values in the manifest for python processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2437: - Assignee: Ferenc Gerlits > Include allowable_values in the manifest for python processors > -- > > Key: MINIFICPP-2437 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2437 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > The {{PropertyDescriptor}} class of Python processors has an > {{allowable_values}} field, but this is not included in the manifest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2441) Use to_array when creating Properties
Ferenc Gerlits created MINIFICPP-2441: - Summary: Use to_array when creating Properties Key: MINIFICPP-2441 URL: https://issues.apache.org/jira/browse/MINIFICPP-2441 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits When creating the static Properties member of Processor and ControllerService objects, we currently use the `std::array` template constructor directly to convert the PropertyDefinition objects to PropertyReference objects, which means we have to manually specify the number of properties. This works, but it isn't elegant. We could use the `std::to_array()` function to do the conversion instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2440) Install all Python dependencies at once
[ https://issues.apache.org/jira/browse/MINIFICPP-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2440: -- Description: When installing dependencies for Python processors, we can get into trouble if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A first, it will install pyfoo==2.0.1, which will cause an error when we try to load processor B. With the current setup, we need to manually resolve the conflict, and (in the example above) add "pyfoo == 1.9.5" to the list of ProcessorDetails.dependencies in processor A if we know we will use it together with processor B. Then, if another processor C comes along with the requirement pyfoo<1.8.0, we will need to change it again, and also update ProcessorDetails.dependencies in processor B. This is not maintainable. Pip is quite smart in resolving such conflicts, but it can only do so if we give it all the requirements at once. So we should collect the dependency requirements from all requirements.txt files and the ProcessorDetails.dependencies fields in all processors, and call pip install with this full list once, instead of calling it multiple times. was: When installing dependencies for Python processors, we can get into trouble if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A first, it will install pyfoo=2.0.1, which will cause an error when we try to load processor B. With the current setup, we need to manually resolve the conflict, and (in the example above) add "pyfoo == 1.9.5" to the list of ProcessorDetails.dependencies in processor A if we know we will use it together with processor B. Then, if another processor C comes along with the requirement pyfoo<1.8.0, we will need to change it again, and also update ProcessorDetails.dependencies in processor B. This is not maintainable. Pip is quite smart in resolving such conflicts, but it can only do so if we give it all the requirements at once. So we should collect the dependency requirements from all requirements.txt files and all ProcessorDetails.dependencies fields in all processors, and call pip install with this full list once, instead of calling it multiple times. > Install all Python dependencies at once > --- > > Key: MINIFICPP-2440 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2440 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Major > > When installing dependencies for Python processors, we can get into trouble > if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends > on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load > processor A first, it will install pyfoo==2.0.1, which will cause an error > when we try to load processor B. > With the current setup, we need to manually resolve the conflict, and (in the > example above) add "pyfoo == 1.9.5" to the list of > ProcessorDetails.dependencies in processor A if we know we will use it > together with processor B. Then, if another processor C comes along with the > requirement pyfoo<1.8.0, we will need to change it again, and also update > ProcessorDetails.dependencies in processor B. This is not maintainable. > Pip is quite smart in resolving such conflicts, but it can only do so if we > give it all the requirements at once. So we should collect the dependency > requirements from all requirements.txt files and the > ProcessorDetails.dependencies fields in all processors, and call pip install > with this full list once, instead of calling it multiple times. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2440) Install all Python dependencies at once
[ https://issues.apache.org/jira/browse/MINIFICPP-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2440: -- Description: When installing dependencies for Python processors, we can get into trouble if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A first, it will install pyfoo=2.0.1, which will cause an error when we try to load processor B. With the current setup, we need to manually resolve the conflict, and (in the example above) add "pyfoo == 1.9.5" to the list of ProcessorDetails.dependencies in processor A if we know we will use it together with processor B. Then, if another processor C comes along with the requirement pyfoo<1.8.0, we will need to change it again, and also update ProcessorDetails.dependencies in processor B. This is not maintainable. Pip is quite smart in resolving such conflicts, but it can only do so if we give it all the requirements at once. So we should collect the dependency requirements from all requirements.txt files and all ProcessorDetails.dependencies fields in all processors, and call pip install with this full list once, instead of calling it multiple times. was: When installing dependencies for Python processors, we can get into trouble if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A first, it will install pyfoo=2.0.1, which will cause an error when we try to load processor B. Pip is quite smart in resolving such conflicts, but it can only do so if we give it all the requirements at once. So we should collect the dependency requirements from all requirements.txt files and all ProcessorDetails.dependencies fields in all processors, and call pip install with this full list once, instead of calling it multiple times. > Install all Python dependencies at once > --- > > Key: MINIFICPP-2440 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2440 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Major > > When installing dependencies for Python processors, we can get into trouble > if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends > on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load > processor A first, it will install pyfoo=2.0.1, which will cause an error > when we try to load processor B. > With the current setup, we need to manually resolve the conflict, and (in the > example above) add "pyfoo == 1.9.5" to the list of > ProcessorDetails.dependencies in processor A if we know we will use it > together with processor B. Then, if another processor C comes along with the > requirement pyfoo<1.8.0, we will need to change it again, and also update > ProcessorDetails.dependencies in processor B. This is not maintainable. > Pip is quite smart in resolving such conflicts, but it can only do so if we > give it all the requirements at once. So we should collect the dependency > requirements from all requirements.txt files and all > ProcessorDetails.dependencies fields in all processors, and call pip install > with this full list once, instead of calling it multiple times. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2440) Install all Python dependencies at once
Ferenc Gerlits created MINIFICPP-2440: - Summary: Install all Python dependencies at once Key: MINIFICPP-2440 URL: https://issues.apache.org/jira/browse/MINIFICPP-2440 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits When installing dependencies for Python processors, we can get into trouble if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A first, it will install pyfoo=2.0.1, which will cause an error when we try to load processor B. Pip is quite smart in resolving such conflicts, but it can only do so if we give it all the requirements at once. So we should collect the dependency requirements from all requirements.txt files and all ProcessorDetails.dependencies fields in all processors, and call pip install with this full list once, instead of calling it multiple times. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2411) Extend NiFi Python API to support source processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2411. --- Fix Version/s: 0.99.1 Resolution: Fixed > Extend NiFi Python API to support source processors > --- > > Key: MINIFICPP-2411 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2411 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.99.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2349) FetchModbusTcp
[ https://issues.apache.org/jira/browse/MINIFICPP-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2349: -- Fix Version/s: 0.99.1 Resolution: Fixed Status: Resolved (was: Patch Available) > FetchModbusTcp > -- > > Key: MINIFICPP-2349 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2349 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.99.1 > > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2428) Add targetUris field support for JSON flow config
[ https://issues.apache.org/jira/browse/MINIFICPP-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2428. --- Fix Version/s: 0.99.1 Resolution: Fixed > Add targetUris field support for JSON flow config > -- > > Key: MINIFICPP-2428 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2428 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.99.1 > > Time Spent: 40m > Remaining Estimate: 0h > > Support for field "targetUri" was in "remoteProcessGroups" was removed in > NiFi and replaced with "targetUris" which is an array. We should also add > support for "targetUris" field and keep the "targetUri" as an alternative for > backwards compatibility. We can use the first value of "targetUris" the same > way we used the value for "targetUri". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2374) Support more distros in the python bootstrapper
[ https://issues.apache.org/jira/browse/MINIFICPP-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2374: -- Fix Version/s: 0.99.1 Resolution: Fixed Status: Resolved (was: Patch Available) > Support more distros in the python bootstrapper > --- > > Key: MINIFICPP-2374 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2374 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Martin Zink >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The python bootstrapper supports these Linux distros: ubuntu, arch, manjaro > and rocky (see > [package_manager.py|https://github.com/apache/nifi-minifi-cpp/blob/main/bootstrap/package_manager.py#L303]). > I think it should support a few more, eg. fedora and debian. The old > boostrap.sh script supports these distros. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2438) MiNiFi logs a failed JSON parsing error after successful C2 flow update
[ https://issues.apache.org/jira/browse/MINIFICPP-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2438. --- Fix Version/s: 0.99.1 Resolution: Fixed > MiNiFi logs a failed JSON parsing error after successful C2 flow update > --- > > Key: MINIFICPP-2438 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2438 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > > In case a flow update happens through C2 server, after the acknowledge > response message is processed MiNiFi logs the following error: > "Failed to parse json response: The document is empty. at 0 ?" > The acknowledge message does not include any payload, so the json parsing > fails in this case. We should ignore any error and should not try to parse > the payload of the acknowledge response. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2437) Include allowable_values in the manifest for python processors
Ferenc Gerlits created MINIFICPP-2437: - Summary: Include allowable_values in the manifest for python processors Key: MINIFICPP-2437 URL: https://issues.apache.org/jira/browse/MINIFICPP-2437 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits The {{PropertyDescriptor}} class of Python processors has an {{allowable_values}} field, but this is not included in the manifest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2436) Allow relative import paths in (NiFi) python processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2436: -- Description: Currently, relative imports like {{from .baseclass import QwertyBase}} (with the dot) in NiFi python processors do not work. Since they do work in NiFi, we should get them to work in MiNiFi C++, as well. It's not as important, but it would be nice if relative import paths worked in old-style python processors, too. was: Currently, relative imports like `from .baseclass import QwertyBase` (with the dot) in NiFi python processors do not work. Since they do work in NiFi, we should get them to work in MiNiFi C++, as well. It's not as important, but it would be nice if relative imports worked in old-style python processors, too. > Allow relative import paths in (NiFi) python processors > --- > > Key: MINIFICPP-2436 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2436 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Currently, relative imports like {{from .baseclass import QwertyBase}} (with > the dot) in NiFi python processors do not work. Since they do work in NiFi, > we should get them to work in MiNiFi C++, as well. > It's not as important, but it would be nice if relative import paths worked > in old-style python processors, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2436) Allow relative import path in (NiFi) python processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2436: -- Summary: Allow relative import path in (NiFi) python processors (was: Allow relative import part in (NiFi) python processors) > Allow relative import path in (NiFi) python processors > -- > > Key: MINIFICPP-2436 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2436 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Currently, relative imports like `from .baseclass import QwertyBase` (with > the dot) in NiFi python processors do not work. Since they do work in NiFi, > we should get them to work in MiNiFi C++, as well. > It's not as important, but it would be nice if relative imports worked in > old-style python processors, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2436) Allow relative import paths in (NiFi) python processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2436: -- Summary: Allow relative import paths in (NiFi) python processors (was: Allow relative import path in (NiFi) python processors) > Allow relative import paths in (NiFi) python processors > --- > > Key: MINIFICPP-2436 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2436 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Currently, relative imports like `from .baseclass import QwertyBase` (with > the dot) in NiFi python processors do not work. Since they do work in NiFi, > we should get them to work in MiNiFi C++, as well. > It's not as important, but it would be nice if relative imports worked in > old-style python processors, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2436) Allow relative import part in (NiFi) python processors
Ferenc Gerlits created MINIFICPP-2436: - Summary: Allow relative import part in (NiFi) python processors Key: MINIFICPP-2436 URL: https://issues.apache.org/jira/browse/MINIFICPP-2436 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Currently, relative imports like `from .baseclass import QwertyBase` (with the dot) in NiFi python processors do not work. Since they do work in NiFi, we should get them to work in MiNiFi C++, as well. It's not as important, but it would be nice if relative imports worked in old-style python processors, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2432) Do not load sample python processors by default
Ferenc Gerlits created MINIFICPP-2432: - Summary: Do not load sample python processors by default Key: MINIFICPP-2432 URL: https://issues.apache.org/jira/browse/MINIFICPP-2432 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits The {{SentimentAnalyzer.py}}, {{ConvertDsToCsv.py}} and {{ExecuteH2oMojoScoring.py}} processors are copied to the {{minifi-python}} directory during installation, so they are loaded when minifi starts. However, this fails since they have python dependencies which are not installed by default. Move these processors to the examples directory -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2431) Custom relationship support in Python processors is broken on Windows (if installed from msi)
[ https://issues.apache.org/jira/browse/MINIFICPP-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2431: -- Description: Custom relationship support in NiFi Python processors requires the new {{relationship.py}} file to be present in the {{minifi-python/nifiapi}} directory, but {{WixWin.msi}} does not copy it there from the source. (was: Custom relationship support requires the new {{relationship.py}} file to be present in the {{minifi-python/nifiapi}} directory, but {{WixWin.msi}} does not copy it there from the source.) > Custom relationship support in Python processors is broken on Windows (if > installed from msi) > - > > Key: MINIFICPP-2431 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2431 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Major > > Custom relationship support in NiFi Python processors requires the new > {{relationship.py}} file to be present in the {{minifi-python/nifiapi}} > directory, but {{WixWin.msi}} does not copy it there from the source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2431) Custom relationship support is broken on Windows (if installed from msi)
Ferenc Gerlits created MINIFICPP-2431: - Summary: Custom relationship support is broken on Windows (if installed from msi) Key: MINIFICPP-2431 URL: https://issues.apache.org/jira/browse/MINIFICPP-2431 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Custom relationship support requires the new {{relationship.py}} file to be present in the {{minifi-python/nifiapi}} directory, but {{WixWin.msi}} does not copy it there from the source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2431) Custom relationship support in NiFi Python processors is broken on Windows (if installed from msi)
[ https://issues.apache.org/jira/browse/MINIFICPP-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2431: -- Summary: Custom relationship support in NiFi Python processors is broken on Windows (if installed from msi) (was: Custom relationship support in Python processors is broken on Windows (if installed from msi)) > Custom relationship support in NiFi Python processors is broken on Windows > (if installed from msi) > -- > > Key: MINIFICPP-2431 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2431 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Major > > Custom relationship support in NiFi Python processors requires the new > {{relationship.py}} file to be present in the {{minifi-python/nifiapi}} > directory, but {{WixWin.msi}} does not copy it there from the source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2431) Custom relationship support in Python processors is broken on Windows (if installed from msi)
[ https://issues.apache.org/jira/browse/MINIFICPP-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2431: -- Summary: Custom relationship support in Python processors is broken on Windows (if installed from msi) (was: Custom relationship support is broken on Windows (if installed from msi)) > Custom relationship support in Python processors is broken on Windows (if > installed from msi) > - > > Key: MINIFICPP-2431 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2431 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Major > > Custom relationship support requires the new {{relationship.py}} file to be > present in the {{minifi-python/nifiapi}} directory, but {{WixWin.msi}} does > not copy it there from the source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2407) minifi.sh: exit on error
[ https://issues.apache.org/jira/browse/MINIFICPP-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2407: -- Fix Version/s: 0.99.1 > minifi.sh: exit on error > > > Key: MINIFICPP-2407 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2407 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2407) minifi.sh: exit on error
[ https://issues.apache.org/jira/browse/MINIFICPP-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2407. --- Resolution: Fixed > minifi.sh: exit on error > > > Key: MINIFICPP-2407 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2407 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2423) Fix RHEL8 bootstrap issues
[ https://issues.apache.org/jira/browse/MINIFICPP-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2423. --- Resolution: Fixed > Fix RHEL8 bootstrap issues > -- > > Key: MINIFICPP-2423 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2423 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.99.1 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2423) Fix RHEL8 bootstrap issues
[ https://issues.apache.org/jira/browse/MINIFICPP-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2423: -- Fix Version/s: 0.99.1 > Fix RHEL8 bootstrap issues > -- > > Key: MINIFICPP-2423 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2423 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Priority: Major > Fix For: 0.99.1 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2423) Fix RHEL8 bootstrap issues
[ https://issues.apache.org/jira/browse/MINIFICPP-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2423: - Assignee: Marton Szasz > Fix RHEL8 bootstrap issues > -- > > Key: MINIFICPP-2423 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2423 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.99.1 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2426) Update the github URL for Librdkafka
[ https://issues.apache.org/jira/browse/MINIFICPP-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2426: -- Description: Librdkafka has been moved from user edenhill to user confluentinc on github: {noformat} $ wget https://github.com/edenhill/librdkafka/archive/v1.6.0.tar.gz --2024-07-11 09:36:45-- https://github.com/edenhill/librdkafka/archive/v1.6.0.tar.gz Resolving github.com (github.com)... 140.82.121.3 Connecting to github.com (github.com)|140.82.121.3|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz [following] --2024-07-11 09:36:45-- https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz Reusing existing connection to github.com:443. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 [following] --2024-07-11 09:36:45-- https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 Resolving codeload.github.com (codeload.github.com)... 140.82.121.10 Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected. HTTP request sent, awaiting response... 200 OK {noformat} (With a few minutes in between when it was not available at either place, giving me a minor heart attack.) Update the download URL in our CMake file. was: Librdkafka has been moved from user edenhill to user confluentinc on github: {noformat} $ wget https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz --2024-07-11 09:30:43-- https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz Resolving github.com (github.com)... 140.82.121.3 Connecting to github.com (github.com)|140.82.121.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 [following] --2024-07-11 09:30:43-- https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 Resolving codeload.github.com (codeload.github.com)... 140.82.121.10 Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected. HTTP request sent, awaiting response... 200 OK {noformat} (With a few minutes in between when it was not available at either place, giving me a minor heart attack.) Update the download URL in our CMake file. > Update the github URL for Librdkafka > > > Key: MINIFICPP-2426 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2426 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Trivial > > Librdkafka has been moved from user edenhill to user confluentinc on github: > {noformat} > $ wget https://github.com/edenhill/librdkafka/archive/v1.6.0.tar.gz > --2024-07-11 09:36:45-- > https://github.com/edenhill/librdkafka/archive/v1.6.0.tar.gz > Resolving github.com (github.com)... 140.82.121.3 > Connecting to github.com (github.com)|140.82.121.3|:443... connected. > HTTP request sent, awaiting response... 301 Moved Permanently > Location: https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz > [following] > --2024-07-11 09:36:45-- > https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz > Reusing existing connection to github.com:443. > HTTP request sent, awaiting response... 302 Found > Location: > https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 > [following] > --2024-07-11 09:36:45-- > https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 > Resolving codeload.github.com (codeload.github.com)... 140.82.121.10 > Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... > connected. > HTTP request sent, awaiting response... 200 OK > {noformat} > (With a few minutes in between when it was not available at either place, > giving me a minor heart attack.) > Update the download URL in our CMake file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2426) Update the github URL for Librdkafka
Ferenc Gerlits created MINIFICPP-2426: - Summary: Update the github URL for Librdkafka Key: MINIFICPP-2426 URL: https://issues.apache.org/jira/browse/MINIFICPP-2426 Project: Apache NiFi MiNiFi C++ Issue Type: Task Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Librdkafka has been moved from user edenhill to user confluentinc on github: {noformat} $ wget https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz --2024-07-11 09:30:43-- https://github.com/confluentinc/librdkafka/archive/v1.6.0.tar.gz Resolving github.com (github.com)... 140.82.121.3 Connecting to github.com (github.com)|140.82.121.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 [following] --2024-07-11 09:30:43-- https://codeload.github.com/confluentinc/librdkafka/tar.gz/refs/tags/v1.6.0 Resolving codeload.github.com (codeload.github.com)... 140.82.121.10 Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected. HTTP request sent, awaiting response... 200 OK {noformat} (With a few minutes in between when it was not available at either place, giving me a minor heart attack.) Update the download URL in our CMake file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reopened MINIFICPP-2405: --- > MemoryUsageTest fails transiently on MacOS in CI > > > Key: MINIFICPP-2405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > MemoryUsageTest fails with the unusual problem that the memory usage is too > low: > {noformat} > 177/264 Test #179: MemoryUsageTest > ..***Failed 0.34 sec > [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::ExtensionManager to trace > [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::Module to trace > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module > 'executable' > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] > Initializing extensions > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing > module 'executable' > Error: 6-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error > in subpattern '': Empty pattern > Randomness seeded to: 507676868 > Physical Memory used by this process: 18251776 bytes > Physical Memory used by the system: 3812900864 bytes > Total Physical Memory in the system: 7516192768 bytes > ~~~ > MemoryUsageTest is a Catch2 v3.4.0 host application. > Run with -? for options > --- > Test Physical memory usage > --- > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 > ... > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: > FAILED: > REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) > with expansion: > 18251776 (0x1168000) > >= > 3000 (0x1c9c380) > === > test cases: 2 | 1 passed | 1 failed > assertions: 2 | 1 passed | 1 failed {noformat} > I have only seen this happen once, and only on MacOS in CI: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2042) Remove (hide) the "Both" option in Output Format of CWEL
[ https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2042: -- Summary: Remove (hide) the "Both" option in Output Format of CWEL (was: Remove (hide) the "Both" Output Address CWEL ambigous Output property ) > Remove (hide) the "Both" option in Output Format of CWEL > > > Key: MINIFICPP-2042 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2042 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Ferenc Gerlits >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > > [~lordgamez] made a great observation > [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182] > {noformat} > Currently we have the options XML, Plaintext, Both, JSON for Output Format. > The "Both" option is quite ambiguous now that we have 3 different output > formats, should we revise this? Either have an "All" option for all 3 output > formats, or rename it to "XML and Plaintext'.{noformat} > I don't see an easy way to fix this without breaking compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2042) Remove (hide) the "Both" Output Address CWEL ambigous Output property
[ https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2042: -- Summary: Remove (hide) the "Both" Output Address CWEL ambigous Output property (was: Address CWEL ambigous Output property ) > Remove (hide) the "Both" Output Address CWEL ambigous Output property > -- > > Key: MINIFICPP-2042 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2042 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Ferenc Gerlits >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > > [~lordgamez] made a great observation > [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182] > {noformat} > Currently we have the options XML, Plaintext, Both, JSON for Output Format. > The "Both" option is quite ambiguous now that we have 3 different output > formats, should we revise this? Either have an "All" option for all 3 output > formats, or rename it to "XML and Plaintext'.{noformat} > I don't see an easy way to fix this without breaking compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2042) Address CWEL ambigous Output property
[ https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2042: -- Fix Version/s: 0.99.1 > Address CWEL ambigous Output property > -- > > Key: MINIFICPP-2042 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2042 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Ferenc Gerlits >Priority: Minor > Fix For: 0.99.1 > > Time Spent: 20m > Remaining Estimate: 0h > > [~lordgamez] made a great observation > [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182] > {noformat} > Currently we have the options XML, Plaintext, Both, JSON for Output Format. > The "Both" option is quite ambiguous now that we have 3 different output > formats, should we revise this? Either have an "All" option for all 3 output > formats, or rename it to "XML and Plaintext'.{noformat} > I don't see an easy way to fix this without breaking compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2422) Remove ubuntu 20.04 support
Ferenc Gerlits created MINIFICPP-2422: - Summary: Remove ubuntu 20.04 support Key: MINIFICPP-2422 URL: https://issues.apache.org/jira/browse/MINIFICPP-2422 Project: Apache NiFi MiNiFi C++ Issue Type: Task Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Remove support for ubuntu 20.04 everywhere, replace with either ubuntu 22.04 or 24.04. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2420) Fix warnings on gcc 13.2
[ https://issues.apache.org/jira/browse/MINIFICPP-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2420: -- Description: There are two warnings when compiling a Release (or RelWithDebInfo) build with gcc 13.2 on Linux, one in JoltUtils.cpp and one in Expression.cpp. As a result, minifi does not compile when MINIFI_FAIL_ON_WARNINGS is set to ON. (was: There are two warnings when compiling with gcc 13.2 on Linux, one in JoltUtils.cpp and one in Expression.cpp. As a result, minifi does not compile when MINIFI_FAIL_ON_WARNINGS is set to ON.) > Fix warnings on gcc 13.2 > > > Key: MINIFICPP-2420 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2420 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > There are two warnings when compiling a Release (or RelWithDebInfo) build > with gcc 13.2 on Linux, one in JoltUtils.cpp and one in Expression.cpp. As a > result, minifi does not compile when MINIFI_FAIL_ON_WARNINGS is set to ON. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2420) Fix warnings on gcc 13.2
Ferenc Gerlits created MINIFICPP-2420: - Summary: Fix warnings on gcc 13.2 Key: MINIFICPP-2420 URL: https://issues.apache.org/jira/browse/MINIFICPP-2420 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits There are two warnings when compiling with gcc 13.2 on Linux, one in JoltUtils.cpp and one in Expression.cpp. As a result, minifi does not compile when MINIFI_FAIL_ON_WARNINGS is set to ON. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2042) Address CWEL ambigous Output property
[ https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2042. --- Resolution: Fixed > Address CWEL ambigous Output property > -- > > Key: MINIFICPP-2042 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2042 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > [~lordgamez] made a great observation > [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182] > {noformat} > Currently we have the options XML, Plaintext, Both, JSON for Output Format. > The "Both" option is quite ambiguous now that we have 3 different output > formats, should we revise this? Either have an "All" option for all 3 output > formats, or rename it to "XML and Plaintext'.{noformat} > I don't see an easy way to fix this without breaking compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2410) AttributesToJSONTests fails transiently in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2410: -- Description: AttributesToJSONTests failed in this test run: [https://github.com/apache/nifi-minifi-cpp/actions/runs/9597199231/job/26465753136?pr=1822] with the message that it failed to parse the JSON string. Unfortunately, it does not log or print the JSON string which failed. We could use the {{CONSERVE(...)}} Catch2 command to debug the issue. The test failure happened on Windows; I don't know if that is relevant. A re-run of the CI job succeeded. (was: AttributesToJSONTests failed in this test run: [https://github.com/apache/nifi-minifi-cpp/actions/runs/9597199231/job/26465753136?pr=1822] with the message that it failed to parse the JSON string. Unfortunately, it does not log or print the JSON string which failed. We could use the {{CONSERVE(...)}} Catch2 command to debug the issue. A re-run of the CI job succeeded.) > AttributesToJSONTests fails transiently in CI > - > > Key: MINIFICPP-2410 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2410 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Priority: Minor > > AttributesToJSONTests failed in this test run: > [https://github.com/apache/nifi-minifi-cpp/actions/runs/9597199231/job/26465753136?pr=1822] > with the message that it failed to parse the JSON string. Unfortunately, it > does not log or print the JSON string which failed. We could use the > {{CONSERVE(...)}} Catch2 command to debug the issue. The test failure > happened on Windows; I don't know if that is relevant. A re-run of the CI > job succeeded. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2410) AttributesToJSONTests fails transiently in CI
Ferenc Gerlits created MINIFICPP-2410: - Summary: AttributesToJSONTests fails transiently in CI Key: MINIFICPP-2410 URL: https://issues.apache.org/jira/browse/MINIFICPP-2410 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits AttributesToJSONTests failed in this test run: [https://github.com/apache/nifi-minifi-cpp/actions/runs/9597199231/job/26465753136?pr=1822] with the message that it failed to parse the JSON string. Unfortunately, it does not log or print the JSON string which failed. We could use the {{CONSERVE(...)}} Catch2 command to debug the issue. A re-run of the CI job succeeded. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2375) ExecuteScript with lua throws if there is no onTrigger function
[ https://issues.apache.org/jira/browse/MINIFICPP-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2375. --- Resolution: Fixed > ExecuteScript with lua throws if there is no onTrigger function > --- > > Key: MINIFICPP-2375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > When running ExecuteScript with > {noformat} > Properties: > Script Engine: lua > Script Body: print("hello world!") {noformat} > onTrigger throws an exception, and the flow file gets penalized, although the > script gets called: "hello world!" is printed to the console. The log file > has > {noformat} > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] > [warning] Caught "attempt to call a nil value > stack traceback:" > (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during > Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 > (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] > [warning] ProcessSession rollback for Execute script executed > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] > [warning] Caught Exception during SchedulingAgent::onTrigger of processor > Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: > N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt > to call a nil value > stack traceback: {noformat} > This is because the LuaScriptEngine requires the {{onTrigger}} function. The > PythonScriptEngine also supports the use case of a simple script without an > {{onTrigger}} function; in this case, no flow files can be processed or > produced, but the processor can still be useful if Timer Driven or Cron > Driven. I think the LuaScriptEngine should support this simple use case, as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2351) MultipartUploadStateStorageTest transiently fails
[ https://issues.apache.org/jira/browse/MINIFICPP-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2351. --- Resolution: Fixed > MultipartUploadStateStorageTest transiently fails > - > > Key: MINIFICPP-2351 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2351 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Failing centos CI job: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/8891797372/job/24414499850] > {noformat} > --- > Remove aged off state > --- > /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:138 > ... > /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:159: > FAILED: > REQUIRE( upload_storage_->getState("test_bucket", "key2") == state2 ) > with expansion: > std::nullopt == {?} > === > test cases: 4 | 3 passed | 1 failed > assertions: 13 | 12 passed | 1 failed > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2405: - Assignee: Ferenc Gerlits > MemoryUsageTest fails transiently on MacOS in CI > > > Key: MINIFICPP-2405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > MemoryUsageTest fails with the unusual problem that the memory usage is too > low: > {noformat} > 177/264 Test #179: MemoryUsageTest > ..***Failed 0.34 sec > [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::ExtensionManager to trace > [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::Module to trace > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module > 'executable' > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] > Initializing extensions > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing > module 'executable' > Error: 6-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error > in subpattern '': Empty pattern > Randomness seeded to: 507676868 > Physical Memory used by this process: 18251776 bytes > Physical Memory used by the system: 3812900864 bytes > Total Physical Memory in the system: 7516192768 bytes > ~~~ > MemoryUsageTest is a Catch2 v3.4.0 host application. > Run with -? for options > --- > Test Physical memory usage > --- > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 > ... > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: > FAILED: > REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) > with expansion: > 18251776 (0x1168000) > >= > 3000 (0x1c9c380) > === > test cases: 2 | 1 passed | 1 failed > assertions: 2 | 1 passed | 1 failed {noformat} > I have only seen this happen once, and only on MacOS in CI: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-2391) Fix ConcurrentQueue move ctor
[ https://issues.apache.org/jira/browse/MINIFICPP-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17855533#comment-17855533 ] Ferenc Gerlits commented on MINIFICPP-2391: --- Fixed by user kraj in pull request #1803. > Fix ConcurrentQueue move ctor > - > > Key: MINIFICPP-2391 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2391 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/nifi-minifi-cpp/pull/1803 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2391) Fix ConcurrentQueue move ctor
[ https://issues.apache.org/jira/browse/MINIFICPP-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2391. --- Resolution: Fixed > Fix ConcurrentQueue move ctor > - > > Key: MINIFICPP-2391 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2391 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/nifi-minifi-cpp/pull/1803 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS
[ https://issues.apache.org/jira/browse/MINIFICPP-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2405: -- Epic Link: MINIFICPP-1324 > MemoryUsageTest fails transiently on MacOS > -- > > Key: MINIFICPP-2405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Priority: Minor > > MemoryUsageTest fails with the unusual problem that the memory usage is too > low: > {noformat} > 177/264 Test #179: MemoryUsageTest > ..***Failed 0.34 sec > [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::ExtensionManager to trace > [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::Module to trace > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module > 'executable' > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] > Initializing extensions > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing > module 'executable' > Error: 6-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error > in subpattern '': Empty pattern > Randomness seeded to: 507676868 > Physical Memory used by this process: 18251776 bytes > Physical Memory used by the system: 3812900864 bytes > Total Physical Memory in the system: 7516192768 bytes > ~~~ > MemoryUsageTest is a Catch2 v3.4.0 host application. > Run with -? for options > --- > Test Physical memory usage > --- > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 > ... > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: > FAILED: > REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) > with expansion: > 18251776 (0x1168000) > >= > 3000 (0x1c9c380) > === > test cases: 2 | 1 passed | 1 failed > assertions: 2 | 1 passed | 1 failed {noformat} > I have only seen this happen once, and only on MacOS in CI: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2405: -- Summary: MemoryUsageTest fails transiently on MacOS in CI (was: MemoryUsageTest fails transiently on MacOS) > MemoryUsageTest fails transiently on MacOS in CI > > > Key: MINIFICPP-2405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Priority: Minor > > MemoryUsageTest fails with the unusual problem that the memory usage is too > low: > {noformat} > 177/264 Test #179: MemoryUsageTest > ..***Failed 0.34 sec > [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::ExtensionManager to trace > [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for > org::apache::nifi::minifi::core::extension::Module to trace > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module > 'executable' > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] > Initializing extensions > [2024-06-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing > module 'executable' > Error: 6-14 12:38:41.435] > [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error > in subpattern '': Empty pattern > Randomness seeded to: 507676868 > Physical Memory used by this process: 18251776 bytes > Physical Memory used by the system: 3812900864 bytes > Total Physical Memory in the system: 7516192768 bytes > ~~~ > MemoryUsageTest is a Catch2 v3.4.0 host application. > Run with -? for options > --- > Test Physical memory usage > --- > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 > ... > /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: > FAILED: > REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) > with expansion: > 18251776 (0x1168000) > >= > 3000 (0x1c9c380) > === > test cases: 2 | 1 passed | 1 failed > assertions: 2 | 1 passed | 1 failed {noformat} > I have only seen this happen once, and only on MacOS in CI: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS
[ https://issues.apache.org/jira/browse/MINIFICPP-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2405: -- Description: MemoryUsageTest fails with the unusual problem that the memory usage is too low: {noformat} 177/264 Test #179: MemoryUsageTest ..***Failed 0.34 sec [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::ExtensionManager to trace [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::Module to trace [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module 'executable' [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] Initializing extensions [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing module 'executable' Error: 6-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error in subpattern '': Empty pattern Randomness seeded to: 507676868 Physical Memory used by this process: 18251776 bytes Physical Memory used by the system: 3812900864 bytes Total Physical Memory in the system: 7516192768 bytes ~~~ MemoryUsageTest is a Catch2 v3.4.0 host application. Run with -? for options --- Test Physical memory usage --- /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 ... /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: FAILED: REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) with expansion: 18251776 (0x1168000) >= 3000 (0x1c9c380) === test cases: 2 | 1 passed | 1 failed assertions: 2 | 1 passed | 1 failed {noformat} I have only seen this happen once, and only on MacOS in CI: [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707] was: MemoryUsageTest fails with the unusual problem that the memory usage is too low: {noformat} 177/264 Test #179: MemoryUsageTest ..***Failed 0.34 sec [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::ExtensionManager to trace [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::Module to trace [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module 'executable' [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] Initializing extensions [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing module 'executable' Error: 6-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error in subpattern '': Empty pattern Randomness seeded to: 507676868 Physical Memory used by this process: 18251776 bytes Physical Memory used by the system: 3812900864 bytes Total Physical Memory in the system: 7516192768 bytes~~~ MemoryUsageTest is a Catch2 v3.4.0 host application. Run with -? for options--- Test Physical memory usage --- /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 .../Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: FAILED: REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) with expansion: 18251776 (0x1168000) >= 3000 (0x1c9c380)=== test cases: 2 | 1 passed | 1 failed assertions: 2 | 1 passed | 1 failed {noformat} I have only seen this happen once, and only on MacOS in CI: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707 > MemoryUsageTest fails transiently on MacOS > -- > > Key: MINIFICPP-2405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter:
[jira] [Created] (MINIFICPP-2405) MemoryUsageTest fails transiently on MacOS
Ferenc Gerlits created MINIFICPP-2405: - Summary: MemoryUsageTest fails transiently on MacOS Key: MINIFICPP-2405 URL: https://issues.apache.org/jira/browse/MINIFICPP-2405 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits MemoryUsageTest fails with the unusual problem that the memory usage is too low: {noformat} 177/264 Test #179: MemoryUsageTest ..***Failed 0.34 sec [2024-06-14 12:38:41.434] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::ExtensionManager to trace [2024-06-14 12:38:41.435] [LogTestController] [info] Setting log level for org::apache::nifi::minifi::core::extension::Module to trace [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Creating module 'executable' [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [trace] Initializing extensions [2024-06-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::Module] [trace] Initializing module 'executable' Error: 6-14 12:38:41.435] [org::apache::nifi::minifi::core::extension::ExtensionManager] [error] Error in subpattern '': Empty pattern Randomness seeded to: 507676868 Physical Memory used by this process: 18251776 bytes Physical Memory used by the system: 3812900864 bytes Total Physical Memory in the system: 7516192768 bytes~~~ MemoryUsageTest is a Catch2 v3.4.0 host application. Run with -? for options--- Test Physical memory usage --- /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:30 .../Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/libminifi/test/unit/MemoryUsageTest.cpp:44: FAILED: REQUIRE( ram_usage_by_process >= gsl::narrow(v.size()) ) with expansion: 18251776 (0x1168000) >= 3000 (0x1c9c380)=== test cases: 2 | 1 passed | 1 failed assertions: 2 | 1 passed | 1 failed {noformat} I have only seen this happen once, and only on MacOS in CI: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/9515743417/job/26230608707 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2343) generateVersion.bat doesn't work if there is a space in the path
[ https://issues.apache.org/jira/browse/MINIFICPP-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2343: - Assignee: Ferenc Gerlits > generateVersion.bat doesn't work if there is a space in the path > > > Key: MINIFICPP-2343 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2343 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2042) Address CWEL ambigous Output property
[ https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2042: - Assignee: Ferenc Gerlits > Address CWEL ambigous Output property > -- > > Key: MINIFICPP-2042 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2042 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Ferenc Gerlits >Priority: Minor > > [~lordgamez] made a great observation > [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182] > {noformat} > Currently we have the options XML, Plaintext, Both, JSON for Output Format. > The "Both" option is quite ambiguous now that we have 3 different output > formats, should we revise this? Either have an "All" option for all 3 output > formats, or rename it to "XML and Plaintext'.{noformat} > I don't see an easy way to fix this without breaking compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2375) ExecuteScript with lua throws if there is no onTrigger function
[ https://issues.apache.org/jira/browse/MINIFICPP-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2375: - Assignee: Ferenc Gerlits > ExecuteScript with lua throws if there is no onTrigger function > --- > > Key: MINIFICPP-2375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When running ExecuteScript with > {noformat} > Properties: > Script Engine: lua > Script Body: print("hello world!") {noformat} > onTrigger throws an exception, and the flow file gets penalized, although the > script gets called: "hello world!" is printed to the console. The log file > has > {noformat} > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] > [warning] Caught "attempt to call a nil value > stack traceback:" > (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during > Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 > (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] > [warning] ProcessSession rollback for Execute script executed > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] > [warning] Caught Exception during SchedulingAgent::onTrigger of processor > Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: > N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt > to call a nil value > stack traceback: {noformat} > This is because the LuaScriptEngine requires the {{onTrigger}} function. The > PythonScriptEngine also supports the use case of a simple script without an > {{onTrigger}} function; in this case, no flow files can be processed or > produced, but the processor can still be useful if Timer Driven or Cron > Driven. I think the LuaScriptEngine should support this simple use case, as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2375) ExecuteScript with lua throws if there is no onTrigger function
[ https://issues.apache.org/jira/browse/MINIFICPP-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2375: -- Description: When running ExecuteScript with {noformat} Properties: Script Engine: lua Script Body: print("hello world!") {noformat} onTrigger throws an exception, and the flow file gets penalized, although the script gets called: "hello world!" is printed to the console. The log file has {noformat} [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] [warning] Caught "attempt to call a nil value stack traceback:" (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] [warning] ProcessSession rollback for Execute script executed [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] [warning] Caught Exception during SchedulingAgent::onTrigger of processor Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt to call a nil value stack traceback: {noformat} This is because the LuaScriptEngine requires the {{onTrigger}} function. The PythonScriptEngine also supports the use case of a simple script without an {{onTrigger}} function; in this case, no flow files can be processed or produced, but the processor can still be useful if Timer Driven or Cron Driven. I think the LuaScriptEngine should support this simple use case, as well. was: When running ExecuteScript with {noformat} Properties: Script Engine: lua Script Body: print("hello world!") {noformat} onTrigger throws an exception, and the flow file gets penalized, although the script gets called: "hello world!" is printed to the console. The log file has {noformat} [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] [warning] Caught "attempt to call a nil value stack traceback:" (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] [warning] ProcessSession rollback for Execute script executed [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] [warning] Caught Exception during SchedulingAgent::onTrigger of processor Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt to call a nil value stack traceback: {noformat} > ExecuteScript with lua throws if there is no onTrigger function > --- > > Key: MINIFICPP-2375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > When running ExecuteScript with > {noformat} > Properties: > Script Engine: lua > Script Body: print("hello world!") {noformat} > onTrigger throws an exception, and the flow file gets penalized, although the > script gets called: "hello world!" is printed to the console. The log file > has > {noformat} > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] > [warning] Caught "attempt to call a nil value > stack traceback:" > (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during > Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 > (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] > [warning] ProcessSession rollback for Execute script executed > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] > [warning] Caught Exception during SchedulingAgent::onTrigger of processor > Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: > N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt > to call a nil value > stack traceback: {noformat} > This is because the LuaScriptEngine requires the {{onTrigger}} function. The > PythonScriptEngine also supports the use case of a simple script without an > {{onTrigger}} function; in this case, no flow files can be processed or > produced, but the processor can still be useful if Timer Driven or Cron > Driven. I think the LuaScriptEngine should support this simple use case, as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2375) ExecuteScript with lua throws if there is no onTrigger function
[ https://issues.apache.org/jira/browse/MINIFICPP-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2375: -- Summary: ExecuteScript with lua throws if there is no onTrigger function (was: ExecuteScript with lua throws ) > ExecuteScript with lua throws if there is no onTrigger function > --- > > Key: MINIFICPP-2375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > When running ExecuteScript with > {noformat} > Properties: > Script Engine: lua > Script Body: print("hello world!") {noformat} > onTrigger throws an exception, and the flow file gets penalized, although the > script gets called: "hello world!" is printed to the console. The log file > has > {noformat} > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] > [warning] Caught "attempt to call a nil value > stack traceback:" > (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during > Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 > (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] > [warning] ProcessSession rollback for Execute script executed > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] > [warning] Caught Exception during SchedulingAgent::onTrigger of processor > Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: > N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt > to call a nil value > stack traceback: {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2375) ExecuteScript with lua throws
[ https://issues.apache.org/jira/browse/MINIFICPP-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2375: -- Description: When running ExecuteScript with {noformat} Properties: Script Engine: lua Script Body: print("hello world!") {noformat} onTrigger throws an exception, and the flow file gets penalized, although the script gets called: "hello world!" is printed to the console. The log file has {noformat} [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] [warning] Caught "attempt to call a nil value stack traceback:" (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] [warning] ProcessSession rollback for Execute script executed [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] [warning] Caught Exception during SchedulingAgent::onTrigger of processor Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt to call a nil value stack traceback: {noformat} was: When running ExecuteScript with {noformat} Properties: Script Engine: python Script Body: print("hello world!") {noformat} onTrigger throws an exception, and the flow file gets penalized, although the script gets called: "hello world!" is printed to the console. The log file has {noformat} [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] [warning] Caught "attempt to call a nil value stack traceback:" (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] [warning] ProcessSession rollback for Execute script executed [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] [warning] Caught Exception during SchedulingAgent::onTrigger of processor Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt to call a nil value stack traceback: {noformat} > ExecuteScript with lua throws > -- > > Key: MINIFICPP-2375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > When running ExecuteScript with > {noformat} > Properties: > Script Engine: lua > Script Body: print("hello world!") {noformat} > onTrigger throws an exception, and the flow file gets penalized, although the > script gets called: "hello world!" is printed to the console. The log file > has > {noformat} > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] > [warning] Caught "attempt to call a nil value > stack traceback:" > (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during > Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 > (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] > [warning] ProcessSession rollback for Execute script executed > [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] > [warning] Caught Exception during SchedulingAgent::onTrigger of processor > Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: > N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt > to call a nil value > stack traceback: {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2351) MultipartUploadStateStorageTest transiently fails
[ https://issues.apache.org/jira/browse/MINIFICPP-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2351: - Assignee: Ferenc Gerlits > MultipartUploadStateStorageTest transiently fails > - > > Key: MINIFICPP-2351 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2351 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Failing centos CI job: > [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/8891797372/job/24414499850] > {noformat} > --- > Remove aged off state > --- > /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:138 > ... > /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:159: > FAILED: > REQUIRE( upload_storage_->getState("test_bucket", "key2") == state2 ) > with expansion: > std::nullopt == {?} > === > test cases: 4 | 3 passed | 1 failed > assertions: 13 | 12 passed | 1 failed > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2384) Update version number to 0.99.1
Ferenc Gerlits created MINIFICPP-2384: - Summary: Update version number to 0.99.1 Key: MINIFICPP-2384 URL: https://issues.apache.org/jira/browse/MINIFICPP-2384 Project: Apache NiFi MiNiFi C++ Issue Type: Task Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2375) ExecuteScript with lua throws
Ferenc Gerlits created MINIFICPP-2375: - Summary: ExecuteScript with lua throws Key: MINIFICPP-2375 URL: https://issues.apache.org/jira/browse/MINIFICPP-2375 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits When running ExecuteScript with {noformat} Properties: Script Engine: python Script Body: print("hello world!") {noformat} onTrigger throws an exception, and the flow file gets penalized, although the script gets called: "hello world!" is printed to the console. The log file has {noformat} [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::Processor] [warning] Caught "attempt to call a nil value stack traceback:" (N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE) during Processor::onTrigger of processor: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5 (Execute script) (2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5) [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::core::ProcessSession] [warning] ProcessSession rollback for Execute script executed [2024-05-15 15:11:29.108] [org::apache::nifi::minifi::SchedulingAgent] [warning] Caught Exception during SchedulingAgent::onTrigger of processor Execute script (uuid: 2cc9c0d6-6ea2-473e-a30d-5debb0b3b2c5), type: N3org6apache4nifi6minifi10extensions3lua18LuaScriptExceptionE, what: attempt to call a nil value stack traceback: {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2374) Support more distros in the python bootstrapper
Ferenc Gerlits created MINIFICPP-2374: - Summary: Support more distros in the python bootstrapper Key: MINIFICPP-2374 URL: https://issues.apache.org/jira/browse/MINIFICPP-2374 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits The python bootstrapper supports these Linux distros: ubuntu, arch, manjaro and rocky (see [package_manager.py|https://github.com/apache/nifi-minifi-cpp/blob/main/bootstrap/package_manager.py#L303]). I think it should support a few more, eg. fedora and debian. The old boostrap.sh script supports these distros. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2351) MultipartUploadStateStorageTest transiently fails
Ferenc Gerlits created MINIFICPP-2351: - Summary: MultipartUploadStateStorageTest transiently fails Key: MINIFICPP-2351 URL: https://issues.apache.org/jira/browse/MINIFICPP-2351 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Failing centos CI job: [https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/8891797372/job/24414499850] {noformat} --- Remove aged off state --- /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:138 ... /opt/minifi/extensions/aws/tests/MultipartUploadStateStorageTest.cpp:159: FAILED: REQUIRE( upload_storage_->getState("test_bucket", "key2") == state2 ) with expansion: std::nullopt == {?} === test cases: 4 | 3 passed | 1 failed assertions: 13 | 12 passed | 1 failed {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-2305) C2VerifyHeartbeatAndStopSecure test transiently fails in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842370#comment-17842370 ] Ferenc Gerlits commented on MINIFICPP-2305: --- Another failure of the same kind on macos: https://github.com/apache/nifi-minifi-cpp/actions/runs/8894224658/job/24422028589?pr=1771 {noformat} Error: 4-30 12:37:34.731] [org::apache::nifi::minifi::core::flow::AdaptiveConfiguration] [error] Error while processing configuration file: Unable to parse configuration file as none of the possible required fields [Flow Controller] is available [in '' section of configuration file] libc++abi: terminating due to uncaught exception of type std::invalid_argument: Unable to parse configuration file as none of the possible required fields [Flow Controller] is available [in '' section of configuration file] {noformat} > C2VerifyHeartbeatAndStopSecure test transiently fails in CI > --- > > Key: MINIFICPP-2305 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2305 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2VerifyHeartbeatAndStopSecure_failure.log > > > C2VerifyHeartbeatAndStopSecure sometimes fails in CI: > {code:java} > Error: 2-23 14:39:39.239] > [org::apache::nifi::minifi::core::flow::AdaptiveConfiguration] [error] Error > while processing configuration file: Unable to parse configuration file as > none of the possible required fields [Flow Controller] is available [in '' > section of configuration file] > terminate called after throwing an instance of 'std::invalid_argument' > what(): Unable to parse configuration file as none of the possible > required fields [Flow Controller] is available [in '' section of > configuration file] {code} > More info in the attached logs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2343) generateVersion.bat doesn't work if there is a space in the path
Ferenc Gerlits created MINIFICPP-2343: - Summary: generateVersion.bat doesn't work if there is a space in the path Key: MINIFICPP-2343 URL: https://issues.apache.org/jira/browse/MINIFICPP-2343 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2330) *.py files are not added to MSI installer if build dir is outside of source root
[ https://issues.apache.org/jira/browse/MINIFICPP-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2330: - Assignee: Ferenc Gerlits > *.py files are not added to MSI installer if build dir is outside of source > root > > > Key: MINIFICPP-2330 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2330 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Assignee: Ferenc Gerlits >Priority: Major > > Python files are added in the wix file with "../" relative path which fails > if the build directory is outside the source root. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (MINIFICPP-2329) Replace the date library with C++20 stdlib functions
[ https://issues.apache.org/jira/browse/MINIFICPP-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837688#comment-17837688 ] Ferenc Gerlits edited comment on MINIFICPP-2329 at 4/16/24 4:18 PM: This works on Visual Studio: https://github.com/fgerlits/nifi-minifi-cpp/commit/4eff2eae66e476cb21da0aac1af650f18a20560d When gcc 14 and llvm 20 (or whichever version supports {{std::chrono::parse}}) are out, we should test this code change with those standard libraries, as well. was (Author: fgerlits): This works on Visual Studio: https://github.com/fgerlits/nifi-minifi-cpp/commit/5764608ccfdc73f7849893eb5da3c36f35b7ec1e When gcc 14 and llvm 20 (or whichever version supports {{std::chrono::parse}}) are out, we should test this code change with those standard libraries, as well. > Replace the date library with C++20 stdlib functions > > > Key: MINIFICPP-2329 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2329 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Everything in Howard Hinnant's date library has been incorporated (either > verbatim or in slightly changed form) in C++20, so we should be able to > remove this dependency and use the new functions in the standard library > instead. > Unfortunately, only Visual Studio has fully implemented this part of C++ 20, > the other standard libraries are still missing {{std::chrono:parse()}}. This > should be fixed in [gcc 14|https://gcc.gnu.org/gcc-14/changes.html] but it's > still not in the upcoming [llvm 19|https://libcxx.llvm.org/ReleaseNotes.html] > release – let's hope for llvm 20. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-2329) Replace the date library with C++20 stdlib functions
[ https://issues.apache.org/jira/browse/MINIFICPP-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837688#comment-17837688 ] Ferenc Gerlits commented on MINIFICPP-2329: --- This works on Visual Studio: https://github.com/fgerlits/nifi-minifi-cpp/commit/5764608ccfdc73f7849893eb5da3c36f35b7ec1e When gcc 14 and llvm 20 (or whichever version supports {{std::chrono::parse}}) are out, we should test this code change with those standard libraries, as well. > Replace the date library with C++20 stdlib functions > > > Key: MINIFICPP-2329 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2329 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Everything in Howard Hinnant's date library has been incorporated (either > verbatim or in slightly changed form) in C++20, so we should be able to > remove this dependency and use the new functions in the standard library > instead. > Unfortunately, only Visual Studio has fully implemented this part of C++ 20, > the other standard libraries are still missing {{std::chrono:parse()}}. This > should be fixed in [gcc 14|https://gcc.gnu.org/gcc-14/changes.html] but it's > still not in the upcoming [llvm 19|https://libcxx.llvm.org/ReleaseNotes.html] > release – let's hope for llvm 20. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2329) Replace the date library with C++20 stdlib functions
[ https://issues.apache.org/jira/browse/MINIFICPP-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2329: -- Description: Everything in Howard Hinnant's date library has been incorporated (either verbatim or in slightly changed form) in C++20, so we should be able to remove this dependency and use the new functions in the standard library instead. Unfortunately, only Visual Studio has fully implemented this part of C++ 20, the other standard libraries are still missing {{std::chrono:parse()}}. This should be fixed in [gcc 14|https://gcc.gnu.org/gcc-14/changes.html] but it's still not in the upcoming [llvm 19|https://libcxx.llvm.org/ReleaseNotes.html] release – let's hope for llvm 20. was: Everything in Howard Hinnant's date library has been incorporated (either verbatim or in slightly changed form) in C++20, so we should be able to remove this dependency and use the new functions in the standard library instead. Unfortunately, only Visual Studio has fully implemented this part of C{+}{+}20, the other standard libraries are still missing {{std::chrono:parse()}}. This should be fixed in [gcc (libstdc{+}{+}) 14|[https://gcc.gnu.org/gcc-14/changes.html],] but it's still not in the upcoming [libc{+}{+} 19|https://libcxx.llvm.org/ReleaseNotes.html] release – let's hope for libc{+}{+} 20. > Replace the date library with C++20 stdlib functions > > > Key: MINIFICPP-2329 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2329 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Everything in Howard Hinnant's date library has been incorporated (either > verbatim or in slightly changed form) in C++20, so we should be able to > remove this dependency and use the new functions in the standard library > instead. > Unfortunately, only Visual Studio has fully implemented this part of C++ 20, > the other standard libraries are still missing {{std::chrono:parse()}}. This > should be fixed in [gcc 14|https://gcc.gnu.org/gcc-14/changes.html] but it's > still not in the upcoming [llvm 19|https://libcxx.llvm.org/ReleaseNotes.html] > release – let's hope for llvm 20. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2329) Replace the date library with C++20 stdlib functions
[ https://issues.apache.org/jira/browse/MINIFICPP-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2329: -- Description: Everything in Howard Hinnant's date library has been incorporated (either verbatim or in slightly changed form) in C++20, so we should be able to remove this dependency and use the new functions in the standard library instead. Unfortunately, only Visual Studio has fully implemented this part of C{+}{+}20, the other standard libraries are still missing {{std::chrono:parse()}}. This should be fixed in [gcc (libstdc{+}{+}) 14|[https://gcc.gnu.org/gcc-14/changes.html],] but it's still not in the upcoming [libc{+}{+} 19|https://libcxx.llvm.org/ReleaseNotes.html] release – let's hope for libc{+}{+} 20. was: Everything in Howard Hinnant's date library has been incorporated (either verbatim or in slightly changed form) in C++20, so we should be able to remove this dependency and use the new functions in the standard library instead. Unfortunately, only Visual Studio has fully implemented this part of C++20, the other standard libraries are still missing {{{}std::chrono:parse(){}}}. This should be fixed in [gcc (libstdc++) 14|[https://gcc.gnu.org/gcc-14/changes.html],] but it's still not in the upcoming [libc++ 19|https://libcxx.llvm.org/ReleaseNotes.html] release -- let's hope for libc++ 20. > Replace the date library with C++20 stdlib functions > > > Key: MINIFICPP-2329 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2329 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > Everything in Howard Hinnant's date library has been incorporated (either > verbatim or in slightly changed form) in C++20, so we should be able to > remove this dependency and use the new functions in the standard library > instead. > Unfortunately, only Visual Studio has fully implemented this part of > C{+}{+}20, the other standard libraries are still missing > {{std::chrono:parse()}}. This should be fixed in [gcc (libstdc{+}{+}) > 14|[https://gcc.gnu.org/gcc-14/changes.html],] but it's still not in the > upcoming [libc{+}{+} 19|https://libcxx.llvm.org/ReleaseNotes.html] release – > let's hope for libc{+}{+} 20. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2329) Replace the date library with C++20 stdlib functions
Ferenc Gerlits created MINIFICPP-2329: - Summary: Replace the date library with C++20 stdlib functions Key: MINIFICPP-2329 URL: https://issues.apache.org/jira/browse/MINIFICPP-2329 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Everything in Howard Hinnant's date library has been incorporated (either verbatim or in slightly changed form) in C++20, so we should be able to remove this dependency and use the new functions in the standard library instead. Unfortunately, only Visual Studio has fully implemented this part of C++20, the other standard libraries are still missing {{{}std::chrono:parse(){}}}. This should be fixed in [gcc (libstdc++) 14|[https://gcc.gnu.org/gcc-14/changes.html],] but it's still not in the upcoming [libc++ 19|https://libcxx.llvm.org/ReleaseNotes.html] release -- let's hope for libc++ 20. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2326) Link lualib statically
[ https://issues.apache.org/jira/browse/MINIFICPP-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2326: -- Description: Installing the liblua library on Windows and using it during the build is a pain. Instead of this, we should bundle liblua and link it statically into the lua script extension DLL. This will make things simpler on Linux, as well. was: Installing the liblua library on Windows and using it during the build is a pain. Instead of this, we should bundle liblua and link it statically into the lua script extension DLL. This will make thing simpler on Linux, as well. > Link lualib statically > -- > > Key: MINIFICPP-2326 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2326 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Installing the liblua library on Windows and using it during the build is a > pain. > Instead of this, we should bundle liblua and link it statically into the lua > script extension DLL. > This will make things simpler on Linux, as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2326) Link lualib statically
Ferenc Gerlits created MINIFICPP-2326: - Summary: Link lualib statically Key: MINIFICPP-2326 URL: https://issues.apache.org/jira/browse/MINIFICPP-2326 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Installing the liblua library on Windows and using it during the build is a pain. Instead of this, we should bundle liblua and link it statically into the lua script extension DLL. This will make thing simpler on Linux, as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2325) Add missing compiler flags to the Ninja build on Windows
Ferenc Gerlits created MINIFICPP-2325: - Summary: Add missing compiler flags to the Ninja build on Windows Key: MINIFICPP-2325 URL: https://issues.apache.org/jira/browse/MINIFICPP-2325 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2324) Add an option to the Windows installer whether to start the service after installation
Ferenc Gerlits created MINIFICPP-2324: - Summary: Add an option to the Windows installer whether to start the service after installation Key: MINIFICPP-2324 URL: https://issues.apache.org/jira/browse/MINIFICPP-2324 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2319) Prevent RepoTests from creating file in the source tree
Ferenc Gerlits created MINIFICPP-2319: - Summary: Prevent RepoTests from creating file in the source tree Key: MINIFICPP-2319 URL: https://issues.apache.org/jira/browse/MINIFICPP-2319 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits A new test was added to RepoTests, which does not set MINIFI_HOME to a temp directory, so it creates the bootstrap.conf file in the source tree. It would be good to figure out how to prevent this from happening in the first place, but for now, I'm just going to fix this one test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2318) Use libc++ in the clang CI build
Ferenc Gerlits created MINIFICPP-2318: - Summary: Use libc++ in the clang CI build Key: MINIFICPP-2318 URL: https://issues.apache.org/jira/browse/MINIFICPP-2318 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits We used to use libc++ as the standard library in the ubuntu_22_04_clang CI build, but this got accidentally lost in some recent change, and now we build with libstdc++. This change reverts to using libc++. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2316) Expression language does not handle ${literal('\n')}
Ferenc Gerlits created MINIFICPP-2316: - Summary: Expression language does not handle ${literal('\n')} Key: MINIFICPP-2316 URL: https://issues.apache.org/jira/browse/MINIFICPP-2316 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits If we put {{{}$\{literal('\n'){ in a property value (which supports expression value) in NiFi, it results in a newline (0x0A) character. In MiNiFi, we get this error message: {noformat} [2024-03-11 10:48:37.164] [org::apache::nifi::minifi::core::Processor] [warning] Caught "1.13: syntax error, unexpected identifier, expecting \ or "'" or "\""" (St13runtime_error) during Processor::onTrigger of processor: 932a0430-3655-4db2-9b4a-bb271ba67d53 (Append newline) (932a0430-3655-4db2-9b4a-bb271ba67d53) [2024-03-11 10:48:37.164] [org::apache::nifi::minifi::core::ProcessSession] [info] Penalizing 7670c84e-df8c-11ee-9667-0242204dc695 for 3ms at Append newline {noformat} Note that the actual property value in {{config.yml}} (or in the exported NiFi config) is {noformat} ${literal('\\n')} {noformat} because the backslash gets escaped. If you manually edit the {{config.yml}} file and enter {{{}$\{literal('\n'){, that works, because we'll have a 0x0A character in the config, so MiNiFi sees {noformat} ${literal(' ')} {noformat} But if you use a tool to generate {{{}config.yml{}}}, it most likely will escape the backslash, so there is no way to have {{$\{literal('\n'){ as the property value. Please check the NiFi code to see the full list of special characters supported by NiFi, and make sure MiNiFi supports the same list. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2310) Extend ListenTCP with new ways to delimit flow files
[ https://issues.apache.org/jira/browse/MINIFICPP-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2310: -- Description: * custom message delimiter * option to separate by connection (no delimiter, only optional size limit) * batching multiple messages to 1 flow file (like NiFi 'Max Batch Size') Also, ListenTCP should not remove the delimiter from the end of the outgoing flow files, to make sure that a byte stream going through PutTCP -> ListenTCP stays the same. (Optionally, we may want to make this configurable.) was: * custom message delimiter * option to separate by connection (no delimiter, only optional size limit) * batching multiple messages to 1 flow file (like NiFi 'Max Batch Size') > Extend ListenTCP with new ways to delimit flow files > > > Key: MINIFICPP-2310 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2310 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marton Szasz >Priority: Major > > * custom message delimiter > * option to separate by connection (no delimiter, only optional size limit) > * batching multiple messages to 1 flow file (like NiFi 'Max Batch Size') > Also, ListenTCP should not remove the delimiter from the end of the outgoing > flow files, to make sure that a byte stream going through PutTCP -> ListenTCP > stays the same. (Optionally, we may want to make this configurable.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2312) Do not return shared_ptrs unnecessarily in ControllerServiceMap and ControllerServiceNode
[ https://issues.apache.org/jira/browse/MINIFICPP-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2312: -- Description: There are several functions in and around Controller Service storage which return shared_ptr's, probably unnecessarily. For example, {noformat} ControllerServiceMap::getAllControllerServices() ControllerServiceNode::getControllerServiceImplementation(); ControllerServiceNode::getLinkedControllerServices(); ControllerServiceNode::getLinkedComponents(); {noformat} Check if these function can return non-owning, possibly const, possibly gsl::not_null, pointers instead of shared_ptrs, and if it is possible, then change them. was: There are several functions in and around Controller Service storage which return shared_ptr's, probably unnecessarily. For example, {noformat} ControllerServiceMap::getAllControllerServices() ControllerServiceNode::getControllerServiceImplementation(); ControllerServiceNode::getLinkedControllerServices(); ControllerServiceNode::getLinkedComponents(); {noformat} Check if these function can return non-owning, possibly const, possibly gsl::non_null, pointers instead of shared_ptrs, and if it is possible, then change them. > Do not return shared_ptrs unnecessarily in ControllerServiceMap and > ControllerServiceNode > - > > Key: MINIFICPP-2312 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2312 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > There are several functions in and around Controller Service storage which > return shared_ptr's, probably unnecessarily. For example, > {noformat} > ControllerServiceMap::getAllControllerServices() > ControllerServiceNode::getControllerServiceImplementation(); > ControllerServiceNode::getLinkedControllerServices(); > ControllerServiceNode::getLinkedComponents(); > {noformat} > Check if these function can return non-owning, possibly const, possibly > gsl::not_null, pointers instead of shared_ptrs, and if it is possible, then > change them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2312) Do not return shared_ptrs unnecessarily in ControllerServiceMap and ControllerServiceNode
[ https://issues.apache.org/jira/browse/MINIFICPP-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2312: -- Description: There are several functions in and around Controller Service storage which return shared_ptr's, probably unnecessarily. For example, {noformat} ControllerServiceMap::getAllControllerServices() ControllerServiceNode::getControllerServiceImplementation(); ControllerServiceNode::getLinkedControllerServices(); ControllerServiceNode::getLinkedComponents(); {noformat} Check if these function can return non-owning, possibly const, possibly gsl::non_null, pointers instead of shared_ptrs, and if it is possible, then change them. was: There are several functions in and around Controller Service strorage which return shared_ptr's, probably unnecessarily. For example, {noformat} ControllerServiceMap::getAllControllerServices() ControllerServiceNode::getControllerServiceImplementation(); ControllerServiceNode::getLinkedControllerServices(); ControllerServiceNode::getLinkedComponents(); {noformat} Check if these function can return non-owning, possibly const, possibly gsl::non_null, pointers instead of shared_ptrs, and if it is possible, then change them. > Do not return shared_ptrs unnecessarily in ControllerServiceMap and > ControllerServiceNode > - > > Key: MINIFICPP-2312 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2312 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > > There are several functions in and around Controller Service storage which > return shared_ptr's, probably unnecessarily. For example, > {noformat} > ControllerServiceMap::getAllControllerServices() > ControllerServiceNode::getControllerServiceImplementation(); > ControllerServiceNode::getLinkedControllerServices(); > ControllerServiceNode::getLinkedComponents(); > {noformat} > Check if these function can return non-owning, possibly const, possibly > gsl::non_null, pointers instead of shared_ptrs, and if it is possible, then > change them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2312) Do not return shared_ptrs unnecessarily in ControllerServiceMap and ControllerServiceNode
Ferenc Gerlits created MINIFICPP-2312: - Summary: Do not return shared_ptrs unnecessarily in ControllerServiceMap and ControllerServiceNode Key: MINIFICPP-2312 URL: https://issues.apache.org/jira/browse/MINIFICPP-2312 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits There are several functions in and around Controller Service strorage which return shared_ptr's, probably unnecessarily. For example, {noformat} ControllerServiceMap::getAllControllerServices() ControllerServiceNode::getControllerServiceImplementation(); ControllerServiceNode::getLinkedControllerServices(); ControllerServiceNode::getLinkedComponents(); {noformat} Check if these function can return non-owning, possibly const, possibly gsl::non_null, pointers instead of shared_ptrs, and if it is possible, then change them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2276) Support FlowFileTransform NiFi Python processors in MiNiFi C++
[ https://issues.apache.org/jira/browse/MINIFICPP-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2276: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Support FlowFileTransform NiFi Python processors in MiNiFi C++ > -- > > Key: MINIFICPP-2276 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2276 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 4h 50m > Remaining Estimate: 0h > > One variant of the NiFi Python processors are inherited from the > FlowFileTransform class which executes a desired transformation on the flow > file content and transfers the results to the "success" relationship in a new > flow file and forwards the original flow file to the "original" relationship. > These kind of python processors should be supported in MiNiFi C++ as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2231) Avoid global CXX flags, especially -Werror
[ https://issues.apache.org/jira/browse/MINIFICPP-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2231. --- Resolution: Fixed > Avoid global CXX flags, especially -Werror > -- > > Key: MINIFICPP-2231 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2231 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > We should only set compiler flags on our own targets, not globally. All > FetchContent dependencies inherit our global settings, but we don't want to > compile our dependencies with Werror enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2278) Add custom relationship support for NiFi python processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2278. --- Resolution: Fixed > Add custom relationship support for NiFi python processors > -- > > Key: MINIFICPP-2278 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2278 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > NiFi python processors have built in "original" and "failure" relationships, > but any other relationship can be specified when transferring the newly > created flowfile. This defaults to the "success" relationship and currently > only the "success" relationship is used in this scenario in the currently > available python processors. > However the dynamically specified relationship support is not present in > MiNiFi C++ as it only supports the "success" relationship at the moment. We > should investigate and implement the support for custom relationships for the > NiFi python processors, also we should investigate if that could be supported > for the native MiNiFi C++ python processors as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2297) Remove iOS support from build system
[ https://issues.apache.org/jira/browse/MINIFICPP-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2297: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Remove iOS support from build system > > > Key: MINIFICPP-2297 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2297 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Trivial > Time Spent: 40m > Remaining Estimate: 0h > > The project was not built for iOS for years and is not maintained at all, > should be removed from the build options. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2303) The Windows CI runner cannot find the Lua library
Ferenc Gerlits created MINIFICPP-2303: - Summary: The Windows CI runner cannot find the Lua library Key: MINIFICPP-2303 URL: https://issues.apache.org/jira/browse/MINIFICPP-2303 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2302) Upgrade to Node.js 20 github actions
Ferenc Gerlits created MINIFICPP-2302: - Summary: Upgrade to Node.js 20 github actions Key: MINIFICPP-2302 URL: https://issues.apache.org/jira/browse/MINIFICPP-2302 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Most of the github user actions we use in our CI jobs use Node.js 16, which is now deprecated, so generates a warning. Upgrade to the latest major versions, which use Node.js 20. Unfortunately, some actions don't have Node.js 20 versions: * seanmiddleditch/gha-setup-ninja@v4 is the latest version, uses Node.js 16 * xpol/setup-lua@v0.3 is the latest version, uses Node.js 12 (!) -- maybe use leafo/gh-actions-lua instead? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2287) Site-to-site with large files: "Site2Site transaction xxx peer unknown respond code 14"
[ https://issues.apache.org/jira/browse/MINIFICPP-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2287. --- Resolution: Duplicate This issue will be fixed by https://issues.apache.org/jira/browse/MINIFICPP-2291, which contains more information about the underlying problem. > Site-to-site with large files: "Site2Site transaction xxx peer unknown > respond code 14" > --- > > Key: MINIFICPP-2287 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2287 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Ferenc Gerlits >Priority: Major > > It looks like nifi may have extended the protocol, and minifi c++ didn't > follow the development. > > From Thomas on the nifi slack: > [https://apachenifi.slack.com/archives/CDF1VC1UZ/p1705327811015419] > > {quote}Running minifi c++ v0.15, I am getting errors when transferring large > files (10gb) via site to site to a Nifi (v1.20) cluster. Per the logs,the > transfer is on going for a while (warning logs, inputPortName has been > running for x ms in \{connection ID} > then it looks like the transfer completes (info log, Site to Site transaction > ... set flow 1 flow records with total size xxx-yyy-zzz ) ALSO, the large > file appears on the remote Nifi cluster > then it looks like the transfer failed (warning log, Site2Site transaction > xxx peer unknown respond code 14) > then another error, (warning log , ProcessSession rollback for inputPortName > executed ) > the finally, (warning protocol transmission failed, yielding ( xxx-yyy-zzz ) > This results in endless copies of the large files as presumably minifi > retries the file despite successfully transferring the file. > The logs show that other smaller files continue to be transferred while the > large files yield. (edited) > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2291) Site-to-site client should treat TRANSACTION_FINISHED_BUT_DESTINATION_FULL as (qualified) success
Ferenc Gerlits created MINIFICPP-2291: - Summary: Site-to-site client should treat TRANSACTION_FINISHED_BUT_DESTINATION_FULL as (qualified) success Key: MINIFICPP-2291 URL: https://issues.apache.org/jira/browse/MINIFICPP-2291 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Currently, the site-to-site client in MiNiFi only accepts {{TRANSACTION_FINISHED(13)}} as a success code, and it treats everything else as failure: https://github.com/apache/nifi-minifi-cpp/blob/9e4ef8d59dbbe345c889edb2f3ed13c98513c96d/libminifi/src/sitetosite/SiteToSiteClient.cpp#L385 {code:c++} if (code == TRANSACTION_FINISHED) { logger_->log_info("Site2Site transaction {} peer finished transaction", transactionID.to_string()); transaction->_state = TRANSACTION_COMPLETED; return true; } else { logger_->log_warn("Site2Site transaction {} peer unknown respond code {}", transactionID.to_string(), magic_enum::enum_underlying(code)); return false; }{code} On the other hand, the NiFi site-to-site client treats both {{TRANSACTION_FINISHED(13)}} and {{TRANSACTION_FINISHED_BUT_DESTINATION_FULL(14)}} as success codes: https://github.com/apache/nifi/blob/3fcf5f25b4c705f6909d058d81a7b8b6d6a648f1/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/protocol/http/StandardHttpFlowFileServerProtocol.java#L120 {code:java} switch (response) { case CONFIRM_TRANSACTION: logger.debug("{} Confirming transaction. checksum={}", this, explanation); commSession.setChecksum(explanation); commSession.setStatus(Transaction.TransactionState.DATA_EXCHANGED); break; case TRANSACTION_FINISHED: case TRANSACTION_FINISHED_BUT_DESTINATION_FULL: logger.debug("{} Transaction is completed. responseCode={}", this, response); commSession.setStatus(Transaction.TransactionState.TRANSACTION_COMPLETED); break; }{code} https://github.com/apache/nifi/blob/3fcf5f25b4c705f6909d058d81a7b8b6d6a648f1/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/protocol/AbstractFlowFileServerProtocol.java#L371 {code:java} logger.debug("{} received {} from {}", new Object[]{this, transactionResponse, peer}); if (transactionResponse.getCode() == ResponseCode.TRANSACTION_FINISHED_BUT_DESTINATION_FULL) { peer.penalize(port.getIdentifier(), port.getYieldPeriod(TimeUnit.MILLISECONDS)); } else if (transactionResponse.getCode() != ResponseCode.TRANSACTION_FINISHED) { throw new ProtocolException("After sending data, expected TRANSACTION_FINISHED response but got " + transactionResponse); }{code} {{TRANSACTION_FINISHED_BUT_DESTINATION_FULL(14)}} can happen when the number or the total size of the files in the batch exceed the limits set in the Back Pressure settings of the outgoing connection of the input port on the server side. MiNiFi should behave in the same way as NiFi: accept {{TRANSACTION_FINISHED_BUT_DESTINATION_FULL(14)}} as success, but the remote processor group should yield, so the server has time to clear the connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2290) Support the default schema when serializing a JSON config
[ https://issues.apache.org/jira/browse/MINIFICPP-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-2290: -- Summary: Support the default schema when serializing a JSON config (was: Handle the default schema correctly when serializing a JSON config) > Support the default schema when serializing a JSON config > - > > Key: MINIFICPP-2290 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2290 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Major > > The JSON flow configuration comes in two flavors: the default schema and the > NiFi schema. In the default schema, the root node location is {{"."}}, which > is a notation borrowed from yaml-cpp to refer to the current node (so > {{node["."] == node}}). > This notation is not supported directly by rapidjson; it works when parsing > the JSON input because it is implemented in {{flow::Node::operator[]}}. > We need to handle {{node["."]}} when writing the JSON flow configuration, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2290) Handle the default schema correctly when serializing a JSON config
Ferenc Gerlits created MINIFICPP-2290: - Summary: Handle the default schema correctly when serializing a JSON config Key: MINIFICPP-2290 URL: https://issues.apache.org/jira/browse/MINIFICPP-2290 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits The JSON flow configuration comes in two flavors: the default schema and the NiFi schema. In the default schema, the root node location is {{"."}}, which is a notation borrowed from yaml-cpp to refer to the current node (so {{node["."] == node}}). This notation is not supported directly by rapidjson; it works when parsing the JSON input because it is implemented in {{flow::Node::operator[]}}. We need to handle {{node["."]}} when writing the JSON flow configuration, too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2287) Site-to-site with large files: "Site2Site transaction xxx peer unknown respond code 14"
[ https://issues.apache.org/jira/browse/MINIFICPP-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2287: - Assignee: Ferenc Gerlits > Site-to-site with large files: "Site2Site transaction xxx peer unknown > respond code 14" > --- > > Key: MINIFICPP-2287 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2287 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Ferenc Gerlits >Priority: Major > > It looks like nifi may have extended the protocol, and minifi c++ didn't > follow the development. > > From Thomas on the nifi slack: > [https://apachenifi.slack.com/archives/CDF1VC1UZ/p1705327811015419] > > {quote}Running minifi c++ v0.15, I am getting errors when transferring large > files (10gb) via site to site to a Nifi (v1.20) cluster. Per the logs,the > transfer is on going for a while (warning logs, inputPortName has been > running for x ms in \{connection ID} > then it looks like the transfer completes (info log, Site to Site transaction > ... set flow 1 flow records with total size xxx-yyy-zzz ) ALSO, the large > file appears on the remote Nifi cluster > then it looks like the transfer failed (warning log, Site2Site transaction > xxx peer unknown respond code 14) > then another error, (warning log , ProcessSession rollback for inputPortName > executed ) > the finally, (warning protocol transmission failed, yielding ( xxx-yyy-zzz ) > This results in endless copies of the large files as presumably minifi > retries the file despite successfully transferring the file. > The logs show that other smaller files continue to be transferred while the > large files yield. (edited) > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12617) Change the default nifi.web.https.host value from 127.0.0.1 to a host name
Ferenc Gerlits created NIFI-12617: - Summary: Change the default nifi.web.https.host value from 127.0.0.1 to a host name Key: NIFI-12617 URL: https://issues.apache.org/jira/browse/NIFI-12617 Project: Apache NiFi Issue Type: Improvement Reporter: Ferenc Gerlits As of NiFi 2.0.0-M1, it is no longer possible to connect to NiFi on {{{}[https://127.0.0.1:8443|https://127.0.0.1:8443/]{}}}, but the default {{nifi.properties}} file still contains {noformat} nifi.web.https.host=127.0.0.1{noformat} and (therefore?) the log file still says {noformat} 2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2024-01-16 14:21:26,375 INFO [main] org.apache.nifi.web.server.JettyServer https://127.0.0.1:8443/nifi{noformat} The default could be changed from {{127.0.0.1}} to (for example) {{{}localhost{}}}, to make the life of novice users easier, and to reduce the number of complaints coming from us. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (MINIFICPP-2229) Encrypt sensitive properties in the flow definition
[ https://issues.apache.org/jira/browse/MINIFICPP-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reopened MINIFICPP-2229: --- > Encrypt sensitive properties in the flow definition > --- > > Key: MINIFICPP-2229 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2229 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Minor > Fix For: 0.16.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > The flow definition (config.yml) file can already be encrypted, but this is > optional. Sensitive properties should always be written to the flow > definition file in encrypted form. All processor and controller service > properties should be marked with a flag that shows if they are sensitive. > If full-flow-definition-encryption is enabled, the flow definition file as a > whole can also be encrypted on top of this (as it is now), but the two > encryptions should use separate keys. -- This message was sent by Atlassian Jira (v8.20.10#820010)