[jira] [Commented] (FLUME-3143) Fix undeclared and unused dependencies
[
https://issues.apache.org/jira/browse/FLUME-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134964#comment-16134964
]
Attila Simon commented on FLUME-3143:
-
Removing the "unused" slf4j-log4j12 dependency from the flume-ng-sdk module
would result in empty test output files. A logger backend implementation
slf4j-log4j12 is a runtime resolved dependency. AFAK dependency analysis only
checks statically the compiled bytecode thus it cannot be perfect around
dependencies used via runtime/reflection (and causing false positives which
would prevent enforcement).
However I recommended to ignoreNonCompile (since the check cannot be perfect
for those dependencies anyway) but in the end it is implementer's call to use
that recommendation or put the known to be runtime to the "exclude from
analysis" list.
If we don't want to enforce the good state at the end then I would see less
value doing this cleanup.
> Fix undeclared and unused dependencies
> --
>
> Key: FLUME-3143
> URL: https://issues.apache.org/jira/browse/FLUME-3143
> Project: Flume
> Issue Type: Bug
>Affects Versions: 1.7.0
>Reporter: Ferenc Szabo
>Assignee: Ferenc Szabo
> Labels: dependency
> Fix For: 1.8.0
>
>
> Dependency analysis shows some warnings in the project that should be fixed
> {noformat}
> [INFO]
>
> [INFO] Building Flume NG SDK 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]commons-lang:commons-lang:jar:2.5:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Configuration 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume Auth 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.thrift:libthrift:jar:0.9.0:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Core 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.code.findbugs:jsr305:jar:1.3.9:compile
> [WARNING]org.apache.httpcomponents:httpcore:jar:4.2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [WARNING]log4j:log4j:jar:1.2.17:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG HDFS Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.4.0:test
> [WARNING]org.apache.flume:flume-ng-auth:jar:1.8.0-SNAPSHOT:compile
> [WARNING]org.apache.avro:avro:jar:1.7.4:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-minicluster:jar:2.4.0:test
> [WARNING]org.apache.hadoop:hadoop-auth:jar:2.4.0:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG IRC Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.guava:guava:jar:11.0.2:compile
> [WARNING]commons-io:commons-io:jar:2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
[jira] [Commented] (FLUME-3143) Fix undeclared and unused dependencies
[
https://issues.apache.org/jira/browse/FLUME-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134245#comment-16134245
]
Ferenc Szabo commented on FLUME-3143:
-
Please show a specific example where the report is wrong. If the slf4j
artifact is used by the tests and listed as compile, then it is unused from the
final jars point of view.
ignoreNonCompile: Ignore Runtime/Provided/Test/System scopes for unused
dependency analysis.
In my understanding it would just not list unused dependencies declared
anything else than compile scope. This might make the analysis faster, however
it would let the non compile scopes to have unused dependencies.
> Fix undeclared and unused dependencies
> --
>
> Key: FLUME-3143
> URL: https://issues.apache.org/jira/browse/FLUME-3143
> Project: Flume
> Issue Type: Bug
>Affects Versions: 1.7.0
>Reporter: Ferenc Szabo
>Assignee: Ferenc Szabo
> Labels: dependency
> Fix For: 1.8.0
>
>
> Dependency analysis shows some warnings in the project that should be fixed
> {noformat}
> [INFO]
>
> [INFO] Building Flume NG SDK 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]commons-lang:commons-lang:jar:2.5:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Configuration 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume Auth 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.thrift:libthrift:jar:0.9.0:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Core 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.code.findbugs:jsr305:jar:1.3.9:compile
> [WARNING]org.apache.httpcomponents:httpcore:jar:4.2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [WARNING]log4j:log4j:jar:1.2.17:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG HDFS Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.4.0:test
> [WARNING]org.apache.flume:flume-ng-auth:jar:1.8.0-SNAPSHOT:compile
> [WARNING]org.apache.avro:avro:jar:1.7.4:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-minicluster:jar:2.4.0:test
> [WARNING]org.apache.hadoop:hadoop-auth:jar:2.4.0:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG IRC Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.guava:guava:jar:11.0.2:compile
> [WARNING]commons-io:commons-io:jar:2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG JDBC channel 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
[jira] [Commented] (FLUME-3143) Fix undeclared and unused dependencies
[
https://issues.apache.org/jira/browse/FLUME-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133987#comment-16133987
]
Attila Simon commented on FLUME-3143:
-
This output seems partly incorrect since it lists slf4j-log4j12 as unused but
slf4j-log4j12 is used in generating the test log thus it's declaration in the
pom is wrong it should have been a runtime only scope eg test. With that said
{{mvn dependency:analyze -DignoreNonCompile}} might be a better way to run this
report. I don't know whether that would miss some other misconfiguration.
Once the cleanup is complete we may want to enforce the correct state as part
of our build:
https://maven.apache.org/plugins/maven-dependency-plugin/analyze-only-mojo.html
> Fix undeclared and unused dependencies
> --
>
> Key: FLUME-3143
> URL: https://issues.apache.org/jira/browse/FLUME-3143
> Project: Flume
> Issue Type: Bug
>Affects Versions: 1.7.0
>Reporter: Ferenc Szabo
>Assignee: Ferenc Szabo
> Labels: dependency
> Fix For: 1.8.0
>
>
> Dependency analysis shows some warnings in the project that should be fixed
> {noformat}
> [INFO]
>
> [INFO] Building Flume NG SDK 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]commons-lang:commons-lang:jar:2.5:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Configuration 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume Auth 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.thrift:libthrift:jar:0.9.0:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG Core 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.code.findbugs:jsr305:jar:1.3.9:compile
> [WARNING]org.apache.httpcomponents:httpcore:jar:4.2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [WARNING]log4j:log4j:jar:1.2.17:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:compile
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG HDFS Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.4.0:test
> [WARNING]org.apache.flume:flume-ng-auth:jar:1.8.0-SNAPSHOT:compile
> [WARNING]org.apache.avro:avro:jar:1.7.4:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.apache.hadoop:hadoop-minicluster:jar:2.4.0:test
> [WARNING]org.apache.hadoop:hadoop-auth:jar:2.4.0:compile
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG IRC Sink 1.8.0-SNAPSHOT
> [INFO]
>
> [WARNING] Used undeclared dependencies found:
> [WARNING]com.google.guava:guava:jar:11.0.2:compile
> [WARNING]commons-io:commons-io:jar:2.1:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]org.slf4j:slf4j-log4j12:jar:1.6.1:test
> [INFO]
>
> [INFO]
>
> [INFO] Building Flume NG JDBC channel 1.8.0-SNAPSHOT
> [INFO]
>
