[jira] [Updated] (MINIFICPP-2346) Speed up MiNiFi Build with Conan package manager (C++)
[
https://issues.apache.org/jira/browse/MINIFICPP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Guzman (Medel) updated MINIFICPP-2346:
Description:
*NOTE:* I noticed you guys recently switched from 0.15 to 0.99, so at certain
point, I will pull in lastest updates from MiNiFi C++ main branch on parent
level and rebase or merge into my PR.
*Description of Issue:* Building MiNiFi C++ with just CMake is really slow.
During the the build process, it clones all the external dependencies and then
builds those external dependencies while its building MiNiFi CPP. Sometimes the
MiNiFi CPP build fails too when external lib dependencies fail to download or
fail to build. It would be faster to build MiNiFi C++ with the external
dependencies already preinstalled.
*Potential Solution:* Integrate *conan version 2* into MiNiFi C++ project to
enable support for using *conan install* to install all or most of MiNiFi's
external dependencies, using *conan install --build=missing* to build them as
prebuilt binary conan packages, upload them to conancenter with {*}conan
upload{*}, and then run cmake generate to generate the appropriate build files
for the OS, then run make to build MiNiFi CPP. Also conan has really good
support for cross platform compatibility for Linux and Windows. At this point
because we already have most of MiNiFi CPP's external lib dependencies
installed with conan, our CMake build will just focus on building the MiNiFi
CPP code, enabling faster builds. Also I will need to account for backward
compatibility support with the previous way of installing MiNiFi CPP external
lib dependencies using CMake FetchContent_Declare(...) approach until
conanfile.py approach covers installing all these dependencies. A valid
verification of the MiNiFi C++ build would be to run GTESTs and if all the
expected test cases pass, then it may be fair to conclude we would be ready to
deprecate the slow CMake FetchContent_Declare(...) approach.
*Steps to Perform MiNiFi C++ Build Enhancement (Plan to Integrate into
script(s)):*
# Install conan version 2 into MiNiFi C++ dev environment (my preference is
using docker container)
# Create a conanfile.py file at root of MiNiFi C++ project where we run "conan
install" or "conan install --build=missing" to install MiNiFi C++ external lib
dependencies first. Then we run CMake to build MiNiFi C++.
# Find all MiNiFi C++ find_package(...) CMake function calls and check if
there is already supported conan packages on the conancenter, so we can install
those packages.
# Find all FetchContent_Declare(...) CMake function calls and check if there
is already supported conan packages on the conancenter, so we can install those
packages.
# At this point we will have used conan version 2 C++ package manager to
install almost all MiNiFi C++'s external dependencies.
## With Conan, we will have installed these MiNiFi external lib dependencies
as prebuilt binary conan packages.
## Or we will have instructed conan to first build each C++ external lib
dependency, upload each lib to conancenter.
# Thus, conan manages installing MiNiFi C++'s external lib dependencies, so we
can then run CMake to build MiNiFi faster.
Here is my draft PR for this Jira Ticket for your reference:
[https://github.com/apache/nifi-minifi-cpp/pull/1775]
Here are the commands that I run:
{code:java}
cd $HOME/src/pipeline/nifi-minifi-cpp
# NOTE: I also plan to add smoother integration of conan commands into MiNiFI
CPP too
# For instance, I know you guys have your bootstrap scripts too
# You also have your github actions workflow scripts too
conan install . --output-folder=build
-pr=$HOME/src/nifi-minifi-cpp/etc/build/conan/profiles/release-linux
# install conan packages and build C++ for minifi using Conan & CMake, we run
this way, so conan's tc.cache_variables
# are like passing -D{minifi_option} upon running cmake generate, then make
to build MiNiFi C++
conan build . --output-folder=build
-pr=/home/bizon/src/jam-repo/main/etc/build/conan/profiles/release-linux{code}
*UPDATE (May 11, 2024)* -
[c96d430|https://github.com/apache/nifi-minifi-cpp/pull/1775/commits/c96d430814a067a6a538f4324337b719cb75bf48]:
Refactoring MiNiFi C++ build infrastructure adding Conan Packager support
*Enabled JNI, SENSORS, USB_CAMERA, OPENCV, and SFTP,* which required creating
*Maven 3.9.6 conan package* updating the conan recipe to expose the Maven
Executable environment variable in CMake and *installing prebuilt Java OpenJDK
21.0.2 conan package* and creating a python wrapper run_maven.py script for
calling the conan maven Java build tool to successfully build Java SFTP Test
Server for SFTP extension testing and maven to build {*}Apache NiFi 1.9.0 JNI
Assembly and Framework{*}; We had to go the python maven wrapper script
approach since I ran into some issues calling conan maven
[jira] [Updated] (NIFI-13216) Replace deprecated GzipUtils methods isCompressedFilename and getUncompressedFilename with the API suggested alternatives
[
https://issues.apache.org/jira/browse/NIFI-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-13216:
Fix Version/s: 2.0.0-M3
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Replace deprecated GzipUtils methods isCompressedFilename and
> getUncompressedFilename with the API suggested alternatives
> -
>
> Key: NIFI-13216
> URL: https://issues.apache.org/jira/browse/NIFI-13216
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Fix For: 2.0.0-M3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> These methods need to be changed in
> {code:java}
> c2/c2-client-bundle/c2-client-service/src/main/java/org/apache/nifi/c2/client/service/operation/TransferDebugOperationHandler.java
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13214) Replace use of deprecated org.apache.lucene.index.IndexReader document methods with API suggestion
[ https://issues.apache.org/jira/browse/NIFI-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13214: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Replace use of deprecated org.apache.lucene.index.IndexReader document > methods with API suggestion > -- > > Key: NIFI-13214 > URL: https://issues.apache.org/jira/browse/NIFI-13214 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > Files which are using the deprecated document methods > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/lucene/DocsReader.java > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/index/lucene/QueryTask.java > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/index/lucene/LuceneEventIndex.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13185) Remove "integration tests" which are not maintained and require personal credentials to Cloud services
[ https://issues.apache.org/jira/browse/NIFI-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13185: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Remove "integration tests" which are not maintained and require personal > credentials to Cloud services > -- > > Key: NIFI-13185 > URL: https://issues.apache.org/jira/browse/NIFI-13185 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > There is a rather large chunk of code and pom configuration sitting in our > codebase for tests which are never run or maintained. These are for cloud > services/integrations that are simply not exercised/maintained and are > therefore not useful. Of course having real tests to these systems is > valuable but unless they're run and maintained they're not. Meanwhile a ton > of excellent work has occurred to vet behaviors using things like test > containers and those are easily run by anyone with docker/etc.. > We should remove all these tests which dont get maintained and require > special permissions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13215) Update AWS, OpenTelemetry, latest dependency scan findings
[ https://issues.apache.org/jira/browse/NIFI-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13215: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Update AWS, OpenTelemetry, latest dependency scan findings > -- > > Key: NIFI-13215 > URL: https://issues.apache.org/jira/browse/NIFI-13215 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13215) Update AWS, OpenTelemetry, latest dependency scan findings
[ https://issues.apache.org/jira/browse/NIFI-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845653#comment-17845653 ] ASF subversion and git services commented on NIFI-13215: Commit 6fc374b268726326a23a3914e0baa78abae0e514 in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6fc374b268 ] NIFI-13215 AWS SDK and Elasticsearch Dependencies Upgraded - Upgraded AWS SDK 1 from 1.12.718 to 1.12.719 - Upgraded AWS SDK 2 from 2.25.48 to 2.25.49 - Upgraded snowflake-ingest-sdk from 2.1.0 to 2.1.1 - Upgraded org.elasticsearch.client from 8.13.3 to 8.13.4 - Upgraded software.amazon.kinesis from 2.5.8 to 2.6.0 This closes #8814 Signed-off-by: David Handermann > Update AWS, OpenTelemetry, latest dependency scan findings > -- > > Key: NIFI-13215 > URL: https://issues.apache.org/jira/browse/NIFI-13215 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13217) Update JavaDoc on core attributes unaffected by removeAttribute / removeAllAttributes
[
https://issues.apache.org/jira/browse/NIFI-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845657#comment-17845657
]
ASF subversion and git services commented on NIFI-13217:
Commit 15696ad86a2716c20d3f1889daa1f23f6e11298b in nifi's branch
refs/heads/main from EndzeitBegins
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=15696ad86a ]
NIFI-13217 Updated ProcessSession Comments to List Immutable Attributes
This closes #8819
Signed-off-by: David Handermann
> Update JavaDoc on core attributes unaffected by removeAttribute /
> removeAllAttributes
> -
>
> Key: NIFI-13217
> URL: https://issues.apache.org/jira/browse/NIFI-13217
> Project: Apache NiFi
> Issue Type: Task
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> NIFI-13200 introduced a behavioral change to the {{standard}} implementation
> of {{ProcessSession}}.
> The existing JavaDoc of {{ProcessSession}} notes that {{removeAttribute}} and
> {{removeAllAttributes}} do not affect the core attribute {{uuid}}. A similar
> note should be added for the core attributes {{path}} and {{filename}} which
> are exempt as well now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-13185 [nifi]
exceptionfactory closed pull request #8818: NIFI-13185 URL: https://github.com/apache/nifi/pull/8818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13216 Replaced deprecated GzipUtils methods isCompressedFilename and getUncompressedFilename with the API suggested alternatives [nifi]
exceptionfactory closed pull request #8816: NIFI-13216 Replaced deprecated GzipUtils methods isCompressedFilename and getUncompressedFilename with the API suggested alternatives URL: https://github.com/apache/nifi/pull/8816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13214 Replaced use of deprecated org.apache.lucene.index.IndexReader document methods with StoredFields document methods. [nifi]
exceptionfactory closed pull request #8815: NIFI-13214 Replaced use of deprecated org.apache.lucene.index.IndexReader document methods with StoredFields document methods. URL: https://github.com/apache/nifi/pull/8815 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-13216) Replace deprecated GzipUtils methods isCompressedFilename and getUncompressedFilename with the API suggested alternatives
[
https://issues.apache.org/jira/browse/NIFI-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845655#comment-17845655
]
ASF subversion and git services commented on NIFI-13216:
Commit 22109de2ad4516329ba7f7db10758157977646f7 in nifi's branch
refs/heads/main from dan-s1
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=22109de2ad ]
NIFI-13216 Replaced deprecated GzipUtils methods
- Replaced isCompressedFilename and getUncompressedFilename with the API
suggested alternatives
This closes #8816
Signed-off-by: David Handermann
> Replace deprecated GzipUtils methods isCompressedFilename and
> getUncompressedFilename with the API suggested alternatives
> -
>
> Key: NIFI-13216
> URL: https://issues.apache.org/jira/browse/NIFI-13216
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> These methods need to be changed in
> {code:java}
> c2/c2-client-bundle/c2-client-service/src/main/java/org/apache/nifi/c2/client/service/operation/TransferDebugOperationHandler.java
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-13217) Update JavaDoc on core attributes unaffected by removeAttribute / removeAllAttributes
[
https://issues.apache.org/jira/browse/NIFI-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-13217.
-
Fix Version/s: 2.0.0-M3
Resolution: Fixed
> Update JavaDoc on core attributes unaffected by removeAttribute /
> removeAllAttributes
> -
>
> Key: NIFI-13217
> URL: https://issues.apache.org/jira/browse/NIFI-13217
> Project: Apache NiFi
> Issue Type: Task
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> NIFI-13200 introduced a behavioral change to the {{standard}} implementation
> of {{ProcessSession}}.
> The existing JavaDoc of {{ProcessSession}} notes that {{removeAttribute}} and
> {{removeAllAttributes}} do not affect the core attribute {{uuid}}. A similar
> note should be added for the core attributes {{path}} and {{filename}} which
> are exempt as well now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13185) Remove "integration tests" which are not maintained and require personal credentials to Cloud services
[ https://issues.apache.org/jira/browse/NIFI-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845656#comment-17845656 ] ASF subversion and git services commented on NIFI-13185: Commit 03005aa13201f4b140751934799140e4c1a1749f in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=03005aa132 ] NIFI-13185 Removed manual and non-working Integration Tests Removed the entire integration-tests-ci profile and reference in GitHub workflow. Removed all tests which require specific local env configs/cloud credentials/ or simply do not work and thus have been blocked from the build for a long time. Set Disabled annotation on a couple key tests which are by design manual and must be retained. The move to testcontainers has been a dramatic improvement in test reliability/behaviors and should be a model for all such tests going forward. This closes #8818 Signed-off-by: David Handermann > Remove "integration tests" which are not maintained and require personal > credentials to Cloud services > -- > > Key: NIFI-13185 > URL: https://issues.apache.org/jira/browse/NIFI-13185 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > There is a rather large chunk of code and pom configuration sitting in our > codebase for tests which are never run or maintained. These are for cloud > services/integrations that are simply not exercised/maintained and are > therefore not useful. Of course having real tests to these systems is > valuable but unless they're run and maintained they're not. Meanwhile a ton > of excellent work has occurred to vet behaviors using things like test > containers and those are easily run by anyone with docker/etc.. > We should remove all these tests which dont get maintained and require > special permissions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13214) Replace use of deprecated org.apache.lucene.index.IndexReader document methods with API suggestion
[ https://issues.apache.org/jira/browse/NIFI-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845654#comment-17845654 ] ASF subversion and git services commented on NIFI-13214: Commit c66a3cf4ff6b531ed87b65f54fe3f73d5e810f16 in nifi's branch refs/heads/main from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c66a3cf4ff ] NIFI-13214 Replaced deprecated Lucene IndexReader methods with StoredFields This closes #8815 Signed-off-by: David Handermann > Replace use of deprecated org.apache.lucene.index.IndexReader document > methods with API suggestion > -- > > Key: NIFI-13214 > URL: https://issues.apache.org/jira/browse/NIFI-13214 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Files which are using the deprecated document methods > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/lucene/DocsReader.java > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/index/lucene/QueryTask.java > # > nifi-extension-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/index/lucene/LuceneEventIndex.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13217 Adjust JavaDoc of ProcessSession to align with changed standard implementation [nifi]
exceptionfactory closed pull request #8819: NIFI-13217 Adjust JavaDoc of ProcessSession to align with changed standard implementation URL: https://github.com/apache/nifi/pull/8819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13215 [nifi]
exceptionfactory closed pull request #8814: NIFI-13215 URL: https://github.com/apache/nifi/pull/8814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-2177) ARM based CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2177: --- Priority: Major (was: Trivial) > ARM based CI > > > Key: MINIFICPP-2177 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2177 > Project: Apache NiFi MiNiFi C++ > Issue Type: Wish >Reporter: Martin Zink >Priority: Major > > It would be awesome if we had some arm based ci aswell, so we can verify that > the arm builds dont break -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13217 Adjust JavaDoc of ProcessSession to align with standard implementation [nifi]
EndzeitBegins opened a new pull request, #8819: URL: https://github.com/apache/nifi/pull/8819 # Summary Adjusts the JavaDoc of `ProcessSession` to align with the changes introduced to the standard implementation with [NIFI-13200](https://issues.apache.org/jira/browse/NIFI-13200) / #8791. This behavioral change might surprise users thus it seems sensible to adjust the JavaDoc with a notice accordingly. [NIFI-13217](https://issues.apache.org/jira/browse/NIFI-13217) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### UI Contributions - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi). ### Licensing - [x] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [x] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [x] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13217) Update JavaDoc on core attributes unaffected by removeAttribute / removeAllAttributes
endzeit created NIFI-13217:
--
Summary: Update JavaDoc on core attributes unaffected by
removeAttribute / removeAllAttributes
Key: NIFI-13217
URL: https://issues.apache.org/jira/browse/NIFI-13217
Project: Apache NiFi
Issue Type: Task
Reporter: endzeit
Assignee: endzeit
NIFI-13200 introduced a behavioral change to the {{standard}} implementation of
{{ProcessSession}}.
The existing JavaDoc of {{ProcessSession}} notes that {{removeAttribute}} and
{{removeAllAttributes}} do not affect the core attribute {{uuid}}. A similar
note should be added for the core attributes {{path}} and {{filename}} which
are exempt as well now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2346) Speed up MiNiFi Build with Conan package manager (C++)
[
https://issues.apache.org/jira/browse/MINIFICPP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Guzman (Medel) updated MINIFICPP-2346:
Description:
*NOTE:* I noticed you guys recently switched from 0.15 to 0.99, so at certain
point, I will pull in lastest updates from MiNiFi C++ main branch on parent
level and rebase or merge into my PR.
*Description of Issue:* Building MiNiFi C++ with just CMake is really slow.
During the the build process, it clones all the external dependencies and then
builds those external dependencies while its building MiNiFi CPP. Sometimes the
MiNiFi CPP build fails too when external lib dependencies fail to download or
fail to build. It would be faster to build MiNiFi C++ with the external
dependencies already preinstalled.
*Potential Solution:* Integrate *conan version 2* into MiNiFi C++ project to
enable support for using *conan install* to install all or most of MiNiFi's
external dependencies, using *conan install --build=missing* to build them as
prebuilt binary conan packages, upload them to conancenter with {*}conan
upload{*}, and then run cmake generate to generate the appropriate build files
for the OS, then run make to build MiNiFi CPP. Also conan has really good
support for cross platform compatibility for Linux and Windows. At this point
because we already have most of MiNiFi CPP's external lib dependencies
installed with conan, our CMake build will just focus on building the MiNiFi
CPP code, enabling faster builds. Also I will need to account for backward
compatibility support with the previous way of installing MiNiFi CPP external
lib dependencies using CMake FetchContent_Declare(...) approach until
conanfile.py approach covers installing all these dependencies. A valid
verification of the MiNiFi C++ build would be to run GTESTs and if all the
expected test cases pass, then it may be fair to conclude we would be ready to
deprecate the slow CMake FetchContent_Declare(...) approach.
*Steps to Perform MiNiFi C++ Build Enhancement (Plan to Integrate into
script(s)):*
# Install conan version 2 into MiNiFi C++ dev environment (my preference is
using docker container)
# Create a conanfile.py file at root of MiNiFi C++ project where we run "conan
install" or "conan install --build=missing" to install MiNiFi C++ external lib
dependencies first. Then we run CMake to build MiNiFi C++.
# Find all MiNiFi C++ find_package(...) CMake function calls and check if
there is already supported conan packages on the conancenter, so we can install
those packages.
# Find all FetchContent_Declare(...) CMake function calls and check if there
is already supported conan packages on the conancenter, so we can install those
packages.
# At this point we will have used conan version 2 C++ package manager to
install almost all MiNiFi C++'s external dependencies.
## With Conan, we will have installed these MiNiFi external lib dependencies
as prebuilt binary conan packages.
## Or we will have instructed conan to first build each C++ external lib
dependency, upload each lib to conancenter.
# Thus, conan manages installing MiNiFi C++'s external lib dependencies, so we
can then run CMake to build MiNiFi faster.
Here is my draft PR for this Jira Ticket for your reference:
[https://github.com/apache/nifi-minifi-cpp/pull/1775]
Here are the commands that I run:
{code:java}
cd $HOME/src/pipeline/nifi-minifi-cpp
# NOTE: I also plan to add smoother integration of conan commands into MiNiFI
CPP too
# For instance, I know you guys have your bootstrap scripts too
# You also have your github actions workflow scripts too
conan install . --output-folder=build
-pr=$HOME/src/nifi-minifi-cpp/etc/build/conan/profiles/release-linux
# install conan packages and build C++ for minifi using Conan & CMake, we run
this way, so conan's tc.cache_variables
# are like passing -D{minifi_option} upon running cmake generate, then make
to build MiNiFi C++
conan build . --output-folder=build
-pr=/home/bizon/src/jam-repo/main/etc/build/conan/profiles/release-linux{code}
*UPDATE (May 11, 2024)* -
[64dd8ad|https://github.com/apache/nifi-minifi-cpp/pull/1775/commits/64dd8ad14d2ddcea5eaa12da14a0e1b8a7289bb8]:
{*}Enabled Build LUA SCRIPTING, ENCRYPT CONFIG, SPLUNK, ELASTIC SEARCH, TEST
PROCESSORS, GRANFANA LOKI and CONTROLLER by Conan & CMake{*}. For building
these components with conan, the LUA SCRIPTING extension required me updating
its Lua.cmake and Sol2.cmake files, so we can use the conan packages in CMake
when USE_CONAN_PACKAGE condition is true. For lua and sol2 conan packages, I
was able to just install the prebuilt binary conan packages from conancenter
directly since conancenter provided the versions that match with MiNiFi C++'s
standalone CMake approach's lua 5.4.6 and sol2 3.3.0 source built versions.
MiNiFi controller being enabled required updating th
