board report for April 2022
NiFi Community, Thank you all for another strong quarter! ## Description: The mission of NiFi is the creation and maintenance of software related to providing an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi MiNiFi is an edge data collection agent built to seamlessly integrate with and leverage the command and control of NiFi. There are both Java and C++ implementations. Apache NiFi Registry is a centralized registry for key configuration items including flow versions, assets, and extensions for Apache NiFi and Apache MiNiFi. Apache NiFi Nar Maven Plugin is a release artifact used for supporting the NiFi classloader isolation model. Apache NiFi Flow Design System is a theme-able set of high quality UI components and utilities for use across the various Apache NiFi web applications in order to provide a more consistent user experience. ## Issues: There are no issues requiring board attention at this time. ## Membership Data: Apache NiFi was founded 2015-07-14 (7 years ago) There are currently 61 committers and 35 PMC members in this project. The Committer-to-PMC ratio is roughly 8:5. Community changes, past quarter: - No new PMC members. Last addition was David Handermann on 2021-09-15. - Ádám Markovics was added as committer on 2022-03-23 - Paul Grey was added as committer on 2022-03-16 ## Project Activity: NiFi 1.16.0 was released on April 4th 2022. This release includes more than 400 JIRAs including features, improvements, bug fixes, and security related items. This release provides for the progression of NiFi, MiNiFi Java, Stateless NiFi, the NiFi registry, and the associated toolkits to ease deployment and management thereof. We are already well underway toward a 1.16.1 stability release with more than 37 JIRAs landed to update dependencies, fix bugs/regressions, and respond to the complex security environment such as the recent 'Springshell' finding. Even though we often find we aren't exposing vulnerabilities we find increasingly users want such dependencies removed/updated rapidly these days and we're trying to accomodate where reasonable. ## Community Health: The community health remains quite strong. We've added committers and have reasonable pipeline for PMC. We continue to see strong mailing list activity. The users email list is less active though we see that well offset with continued growth and activity in our Slack channels. Our general channel has 2042 users as of the time of this writing. We have a new contributor that has also started to express interest and make progress on revamping the Apache NiFi website which could definitely use a reboot and move us to preferred Apache infrastructure/processes. We also see strong growth in JIRAs and PRs both opened and closed this quarter landing more than 330 commits!
Re: Missing maven dependencies when building nifi
Good point there James. Phil definitely give that a look. Those commands need to be run on the repository at the right time as well so it doesn't import with the wrong line endings. >From our github automated builds we have at: https://github.com/apache/nifi/blob/main/.github/workflows/ci-workflow.yml Here are key things to execute to run like we do there: git config --global core.autocrlf false git config --global core.longpaths true Checkout Code DEFAULT_MAVEN_OPTS: >- -Xmx3g -XX:ReservedCodeCacheSize=1g -XX:+UseG1GC -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false mvn package verify -P contrib-check -P include-grpc -P skip-nifi-bin-assembly On Tue, Apr 5, 2022 at 2:31 PM James Srinivasan wrote: > Possibly not your problem, but did you follow the instructions for building > on Windows here: > > https://nifi.apache.org/quickstart.html > > On Tue, 5 Apr 2022, 12:42 Phil H, wrote: > > > Okay, I started over with a completely new local repo. This is the exact > > sequence of commands I ran: > > > > $ git clone https://github.com/apache/nifi.git > > $ cd nifi > > $ git remote add upstream https://github.com/apache/nifi.git > > $ git clean -fxd > > $ mvn clean package verify -Pcontrib-check,include-grpc > > > > And after about 12 minutes: > > > > [ERROR] Failed to execute goal > > org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test > (default-test) > > on project nifi-web-security: There are test failures. > > [ERROR] > > [ERROR] Please refer to > > > > > D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\surefire-reports > > for the individual test results. > > [ERROR] Please refer to dump files (if any exist) [date].dump, > > [date]-jvmRun[N].dump and [date].dumpstream. > > > > The contents of the dumpfile at that location is: > > > > # Created at 2022-04-05T21:05:18.979 > > Boot Manifest-JAR contains absolute paths in classpath > > > > > 'D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\test-classes' > > Hint: > > -Djdk.net.URLClassPath.disableClassPathURLCheck=true > > > > I'm sure there's some obvious mistake I'm making, or piece of > information I > > am missing, but I have no idea what they are! > > > > Thanks > > Phil > > > > On Tue, Apr 5, 2022 at 12:53 AM Joe Witt wrote: > > > > > Phil > > > > > > I think you need to run 'git clean -fxd' from the nifi source root. > > > > > > Then try the build again. And allow the entire thing to run. > > > > > > Too many builds in different levels are leaving things around. > > > > > > Thanks > > > > > > On Mon, Apr 4, 2022 at 7:46 AM Phil H wrote: > > > > > > > So, about that error that flashed past... > > > > > > > > [INFO] > > > > > > > > > > [INFO] BUILD FAILURE > > > > [INFO] > > > > > > > > > > [INFO] Total time: 22:21 min (Wall Clock) > > > > [INFO] Finished at: 2022-04-05T00:37:46+10:00 > > > > [INFO] > > > > > > > > > > [ERROR] Failed to execute goal > > > org.apache.rat:apache-rat-plugin:0.13:check > > > > (default) on project nifi-poi-processors: Too many files with > > unapproved > > > > license: 1 See RAT report in: > > > > > > > > > > > > > > D:\nifi.github\nifi\nifi-nar-bundles\nifi-poi-bundle\nifi-poi-processors\target\rat.txt > > > > -> [Help 1] > > > > > > > > The rat file contains the following: > > > > > > > > * > > > > Summary > > > > --- > > > > Generated at: 2022-04-05T00:33:56+10:00 > > > > > > > > Notes: 0 > > > > Binaries: 4 > > > > Archives: 0 > > > > Standards: 7 > > > > > > > > Apache Licensed: 6 > > > > Generated Documents: 0 > > > > > > > > JavaDocs are generated, thus a license header is optional. > > > > Generated files do not require license headers. > > > > > > > > 1 Unknown Licenses > > > > > > > > * > > > > > > > > Files with unapproved licenses: > > > > > > > > > > > > > > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls > > > > > > > > * > > > > > > > > * > > > > Files with Apache License headers will be marked AL > > > > Binary files (which do not require any license headers) will be > > marked > > > B > > > > Compressed archives will be marked A > > > > Notices, licenses etc. will be marked N > > > > AL > > > > > > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/pom.xml > > > > AL > > > > > > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-
Re: Missing maven dependencies when building nifi
Possibly not your problem, but did you follow the instructions for building on Windows here: https://nifi.apache.org/quickstart.html On Tue, 5 Apr 2022, 12:42 Phil H, wrote: > Okay, I started over with a completely new local repo. This is the exact > sequence of commands I ran: > > $ git clone https://github.com/apache/nifi.git > $ cd nifi > $ git remote add upstream https://github.com/apache/nifi.git > $ git clean -fxd > $ mvn clean package verify -Pcontrib-check,include-grpc > > And after about 12 minutes: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) > on project nifi-web-security: There are test failures. > [ERROR] > [ERROR] Please refer to > > D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\surefire-reports > for the individual test results. > [ERROR] Please refer to dump files (if any exist) [date].dump, > [date]-jvmRun[N].dump and [date].dumpstream. > > The contents of the dumpfile at that location is: > > # Created at 2022-04-05T21:05:18.979 > Boot Manifest-JAR contains absolute paths in classpath > > 'D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\test-classes' > Hint: > -Djdk.net.URLClassPath.disableClassPathURLCheck=true > > I'm sure there's some obvious mistake I'm making, or piece of information I > am missing, but I have no idea what they are! > > Thanks > Phil > > On Tue, Apr 5, 2022 at 12:53 AM Joe Witt wrote: > > > Phil > > > > I think you need to run 'git clean -fxd' from the nifi source root. > > > > Then try the build again. And allow the entire thing to run. > > > > Too many builds in different levels are leaving things around. > > > > Thanks > > > > On Mon, Apr 4, 2022 at 7:46 AM Phil H wrote: > > > > > So, about that error that flashed past... > > > > > > [INFO] > > > > > > > [INFO] BUILD FAILURE > > > [INFO] > > > > > > > [INFO] Total time: 22:21 min (Wall Clock) > > > [INFO] Finished at: 2022-04-05T00:37:46+10:00 > > > [INFO] > > > > > > > [ERROR] Failed to execute goal > > org.apache.rat:apache-rat-plugin:0.13:check > > > (default) on project nifi-poi-processors: Too many files with > unapproved > > > license: 1 See RAT report in: > > > > > > > > > D:\nifi.github\nifi\nifi-nar-bundles\nifi-poi-bundle\nifi-poi-processors\target\rat.txt > > > -> [Help 1] > > > > > > The rat file contains the following: > > > > > > * > > > Summary > > > --- > > > Generated at: 2022-04-05T00:33:56+10:00 > > > > > > Notes: 0 > > > Binaries: 4 > > > Archives: 0 > > > Standards: 7 > > > > > > Apache Licensed: 6 > > > Generated Documents: 0 > > > > > > JavaDocs are generated, thus a license header is optional. > > > Generated files do not require license headers. > > > > > > 1 Unknown Licenses > > > > > > * > > > > > > Files with unapproved licenses: > > > > > > > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls > > > > > > * > > > > > > * > > > Files with Apache License headers will be marked AL > > > Binary files (which do not require any license headers) will be > marked > > B > > > Compressed archives will be marked A > > > Notices, licenses etc. will be marked N > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/pom.xml > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/java/org/apache/nifi/processors/poi/ConvertExcelToCSVProcessor.java > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/resources/docs/org.apache.nifi.processors.poi.ConvertExcelToCSVProcessor/additionalDetails.html > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/java/org/apache/nifi/processors/poi/ConvertExcelToCSVProcessorTest.java > > > B > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/CollegeScorecard.xlsx > > > B > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/dataformatting.xlsx > > > AL > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-pr
Re: [discuss] nifi 1.16.1
Joe, Thanks for wrapping up the 1.16.0 release and preparing 1.16.1, there are a handful of bug fixes and dependency updates that will be helpful to include as a part of 1.16.1. I am in the process of reviewing NIFI-7253 upgrading Jackson, it looks like it should be ready, pending successful automated builds. The Avro upgrade for NIFI-7234 covers more ground in terms of upgrading versions, and appears to involve more changes. We definitely should move in that direction, and I think there have been some previous efforts to upgrade Avro dependencies. I can take an initial look at the PR soon, but it would be helpful for others to give it more attention. There are a couple other dependency upgrades I plan to put forward soon, so perhaps waiting until Wednesday would give a bit more time to get some of those completed. Regards, David Handermann On Mon, Apr 4, 2022 at 3:45 PM Mike Thomsen wrote: > I have a few PRs for standardizing and updating our dependencies like > Jackson and Avro. Might be good to get those included in 1.16.1 > > On Mon, Apr 4, 2022 at 3:02 PM Joe Witt wrote: > > > > Team, > > > > Sorry for the delays in wrapping up the 1.16 release from earlier last > > week. We had great vote turnout and it was definitely a large release in > > terms of features/etc.. > > > > Watching evolution on the 1.17 line since it is clear we can benefit > from a > > 1.16.1 release so I've started preparing that and will closely monitor > > candidates for inclusion. > > > > The branch is pushed and is called 'support/nifi-1.16'. I've already > > pulled in everything on 1.17 that looks like it can work in a pure 3rd > > digit release. Some of this makes sense due to bugs found and or natural > > concerns of scanners for things like springshell/etc.. In any case it is > > always good to tighten up the release lines if time permits. > > > > I'll probably start the release process tomorrow or Wednesday. Will > watch > > for anything that comes in. > > > > Thanks >
Re: Missing maven dependencies when building nifi
Okay, I started over with a completely new local repo. This is the exact sequence of commands I ran: $ git clone https://github.com/apache/nifi.git $ cd nifi $ git remote add upstream https://github.com/apache/nifi.git $ git clean -fxd $ mvn clean package verify -Pcontrib-check,include-grpc And after about 12 minutes: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project nifi-web-security: There are test failures. [ERROR] [ERROR] Please refer to D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. The contents of the dumpfile at that location is: # Created at 2022-04-05T21:05:18.979 Boot Manifest-JAR contains absolute paths in classpath 'D:\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-web\nifi-web-security\target\test-classes' Hint: -Djdk.net.URLClassPath.disableClassPathURLCheck=true I'm sure there's some obvious mistake I'm making, or piece of information I am missing, but I have no idea what they are! Thanks Phil On Tue, Apr 5, 2022 at 12:53 AM Joe Witt wrote: > Phil > > I think you need to run 'git clean -fxd' from the nifi source root. > > Then try the build again. And allow the entire thing to run. > > Too many builds in different levels are leaving things around. > > Thanks > > On Mon, Apr 4, 2022 at 7:46 AM Phil H wrote: > > > So, about that error that flashed past... > > > > [INFO] > > > > [INFO] BUILD FAILURE > > [INFO] > > > > [INFO] Total time: 22:21 min (Wall Clock) > > [INFO] Finished at: 2022-04-05T00:37:46+10:00 > > [INFO] > > > > [ERROR] Failed to execute goal > org.apache.rat:apache-rat-plugin:0.13:check > > (default) on project nifi-poi-processors: Too many files with unapproved > > license: 1 See RAT report in: > > > > > D:\nifi.github\nifi\nifi-nar-bundles\nifi-poi-bundle\nifi-poi-processors\target\rat.txt > > -> [Help 1] > > > > The rat file contains the following: > > > > * > > Summary > > --- > > Generated at: 2022-04-05T00:33:56+10:00 > > > > Notes: 0 > > Binaries: 4 > > Archives: 0 > > Standards: 7 > > > > Apache Licensed: 6 > > Generated Documents: 0 > > > > JavaDocs are generated, thus a license header is optional. > > Generated files do not require license headers. > > > > 1 Unknown Licenses > > > > * > > > > Files with unapproved licenses: > > > > > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls > > > > * > > > > * > > Files with Apache License headers will be marked AL > > Binary files (which do not require any license headers) will be marked > B > > Compressed archives will be marked A > > Notices, licenses etc. will be marked N > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/pom.xml > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/java/org/apache/nifi/processors/poi/ConvertExcelToCSVProcessor.java > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/resources/docs/org.apache.nifi.processors.poi.ConvertExcelToCSVProcessor/additionalDetails.html > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/java/org/apache/nifi/processors/poi/ConvertExcelToCSVProcessorTest.java > > B > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/CollegeScorecard.xlsx > > B > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/dataformatting.xlsx > > AL > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/logback-test.xml > > B > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/TwoSheets.xlsx > > !? > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls > > B > > > > > D:/nifi.github/nifi/nifi-nar-bundles/nifi-poi-bundle/nifi-poi-processors/src/test/resources/with-blank-cells.xlsx > > > > ***
