We should discontinue Kite processors from the build at all in 1.16.0. Ill file a Jira
Thanks On Wed, Jan 19, 2022 at 7:47 AM David Handermann < [email protected]> wrote: > Mike, > > Thanks for the additional details. According to the original output, the > problem appears to be in the nifi-kite-processors module. The > nifi-kite-processors module includes a dependency on hive-hcatalog-core, > the version of which is based on the hive.version property. We should be > able to add an exclusion for log4j-core within that dependency definition, > which will allow it to compile with Hive 3.1.2. For the moment, if you > don't need the Kite processors, you could skip building nifi-kite-nar, or > you could manually add the following exclusion: > > <exclusion> > <groupId>org.apache.logging.log4j</groupId> > <artifactId>log4j-core</artifactId> > <exclusion> > > Regards, > David Handermann > > On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski < > [email protected]> wrote: > >> Compilation with -Dhive.version=3.1.0 makes the problem. >> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 >> is not successful. >> We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive >> site. Also without success. >> >> Please note that without this parameter probably hive2 version is >> compiled into nifi and HiveStreaming is not working then. >> And hive 3 is most popular novadays. >> >> Regards, >> Mike >> >> -----Original Message----- >> From: Bryan Bende <[email protected]> >> Sent: Wednesday, January 19, 2022 3:03 PM >> To: [email protected] >> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> >> We did ban all use of log4j-core in favor of log4j-to-slf4j... >> >> https://issues.apache.org/jira/browse/NIFI-9483 >> >> With your build specifying the -D versions, it created a different >> classpath during your build than what existed during the normal build of >> the release. >> >> We should try to fix any issues like this though, so if you can figure >> out which dependency path is the issue, the fix is to add some exclusions >> and then to ensure that log4j-to-slf4j dependency is added. >> >> On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski < >> [email protected]> wrote: >> > >> > OK, but there is information that log4j-core:jar:2.17.1 is blocked: >> > >> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies >> failed with message: >> > >> > Found Banned Dependency: >> > org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > while this version is not impacted by log4j security vulnerability: >> > >> > https://logging.apache.org/log4j/2.x/security.html >> > >> > >> > >> > So for sure it shouldn’t be banned. >> > >> > >> > >> > Regards, >> > >> > Mike >> > >> > From: Pierre Villard <[email protected]> >> > Sent: Wednesday, January 19, 2022 1:11 PM >> > To: [email protected] >> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> > >> > >> > >> > We recently changed the build to fail if it's including problematic >> transitive dependencies for log4j. I think those are coming with the >> specific versions of HBase/Hive you're using. Not a maven expert but I >> guess there is a way to exclude the step "enforce-banned-dependencies" from >> the build. >> > >> > >> > >> > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski < >> [email protected]> a écrit : >> > >> > Hi Edward, >> > >> > Full mvn dependency:tree log enclosed. >> > >> > We’re compiling using command: >> > >> > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 >> > -Dhbase.version=2.0.2 >> > >> > >> > >> > Regards, >> > >> > Mike >> > >> > >> > >> > From: Edward Armes <[email protected]> >> > Sent: Wednesday, January 19, 2022 12:34 PM >> > To: [email protected] >> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> > >> > >> > >> > Hi Mike, >> > >> > >> > >> > Can you give the full output you get from mvn dependency:tree as we >> 15.2 and 3 was done to remove log4j dependences from Nifi as a safety >> measure due the recent log4shell issues. >> > >> > >> > >> > Thanks >> > >> > >> > >> > Edward >> > >> > >> > >> > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski < >> [email protected]> wrote: >> > >> > Hi, >> > >> > We are trying to compile nifi from sources. >> > >> > There is no problem with 1.15.1 compilation but in case of 1.15.2 and >> 1.15.3 there are compilation errors. >> > >> > >> > >> > Can you suggest how to solve this problem? >> > >> > >> > >> > mvn dependency:tree has no errors: >> > >> > >> > >> > >> > >> > $ tail log1.txt >> > >> > [INFO] nifi-toolkit-flowfile-repo ......................... SUCCESS [ >> > 0.005 s] >> > >> > [INFO] nifi-toolkit-flowanalyzer .......................... SUCCESS [ >> > 0.008 s] >> > >> > [INFO] nifi-toolkit-assembly .............................. SUCCESS [ >> > 0.463 s] >> > >> > [INFO] nifi-toolkit-api ................................... SUCCESS [ >> > 0.023 s] >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [INFO] BUILD SUCCESS >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [INFO] Total time: 02:41 min >> > >> > [INFO] Finished at: 2022-01-19T08:51:46+01:00 >> > >> > >> > >> > >> > >> > Compilation of branch rel/nifi-1.15.3: >> > >> > >> > >> > [DEBUG] Adding ERROR message due to exception >> > >> > org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned >> > Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > Use 'mvn dependency:tree' to locate the source of the banned >> dependencies. >> > >> > at >> > org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute >> > (AbstractBanDependencies.java:113) >> > >> > at org.apache.maven.plugins.enforcer.EnforceMojo.execute >> > (EnforceMojo.java:200) >> > >> > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo >> > (DefaultBuildPluginManager.java:137) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:210) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies >> failed with message: >> > >> > Found Banned Dependency: >> > org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > Use 'mvn dependency:tree' to locate the source of the banned >> dependencies. >> > >> > [INFO] >> > >> > >> > >> > …. >> > >> > >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [ERROR] Failed to execute goal >> > org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce >> > (enforce-banned-dependencies) on project nifi-kite-processors: Some >> > Enforcer rules have failed. Look above for specific messages >> > explaining why the rule failed. -> [Help 1] >> > >> > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to >> execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce >> (enforce-banned-dependencies) on project nifi-kite-processors: Some >> Enforcer rules have failed. Look above for specific messages explaining why >> the rule failed. >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:215) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > Caused by: org.apache.maven.plugin.MojoExecutionException: Some >> Enforcer rules have failed. Look above for specific messages explaining why >> the rule failed. >> > >> > at org.apache.maven.plugins.enforcer.EnforceMojo.execute >> > (EnforceMojo.java:255) >> > >> > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo >> > (DefaultBuildPluginManager.java:137) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:210) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > [ERROR] >> > >> > [ERROR] >> > >> > [ERROR] For more information about the errors and possible solutions, >> please read the following articles: >> > >> > [ERROR] [Help 1] >> > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptio >> > n >> > >> > [ERROR] >> > >> > [ERROR] After correcting the problems, you can resume the build with >> > the command >> > >> > [ERROR] mvn <args> -rf :nifi-kite-processors >> > >> > >> > >> > Regards, >> > >> > Mike >> >
