[
https://issues.apache.org/jira/browse/SPARK-47042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817403#comment-17817403
]
William Wong commented on SPARK-47042:
--------------------------------------
Apart from missing commons-lang3 dependencies, 'commons-utils' module also
missing explicit dependencies for 'jackson-core' and 'jackson-annotations'.
Should I also fix it with this JIRA, or create another Jira for fixing them?
{code:java}
~/dev/sources/spark$ ./build/mvn -T 2C -pl ./common/utils/pom.xml
dependency:analyze
Using `mvn` from path:
/home/william/dev/sources/spark/build/apache-maven-3.9.6/bin/mvn
[INFO] Scanning for projects...
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 24
[INFO]
[INFO] --------------< org.apache.spark:spark-common-utils_2.13 >--------------
[INFO] Building Spark Project Common Utils 4.0.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] >>> dependency:3.6.0:analyze (default-cli) > test-compile @
spark-common-utils_2.13 >>>
[INFO]
[INFO] --- enforcer:3.3.0:enforce (enforce-versions) @ spark-common-utils_2.13
---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireMavenVersion
passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.dependency.BannedDependencies
passed
[INFO] Rule 3:
org.codehaus.mojo.extraenforcer.dependencies.EnforceBytecodeVersion passed
[INFO]
[INFO] --- enforcer:3.3.0:enforce (enforce-no-duplicate-dependencies) @
spark-common-utils_2.13 ---
[INFO] Rule 0:
org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO]
[INFO] --- antrun:3.1.0:run (choose-shell-and-script) @ spark-common-utils_2.13
---
[INFO] Executing tasks
[WARNING] [echo] Shell to use for generating spark-version-info.properties
file =
[WARNING] [echo] bash
[WARNING] [echo] Script to use for generating
spark-version-info.properties file =
[WARNING] [echo] spark-build-info
[INFO] Executed tasks
[INFO]
[INFO] --- scalafmt:1.1.1640084764.9f463a9:format (default) @
spark-common-utils_2.13 ---
[WARNING] format.skipSources set, ignoring main directories
[WARNING] format.skipTestSources set, ignoring validateOnly directories
[WARNING] No sources specified, skipping formatting
[INFO]
[INFO] --- scala:4.7.1:add-source (eclipse-add-source) @
spark-common-utils_2.13 ---
[INFO] Add Source directory:
/home/william/dev/sources/spark/common/utils/src/main/scala
[INFO] Add Test Source directory:
/home/william/dev/sources/spark/common/utils/src/test/scala
[INFO]
[INFO] --- dependency:3.6.0:build-classpath (default-cli) @
spark-common-utils_2.13 ---
[INFO] Dependencies classpath:
.....
[INFO]
[INFO] <<< dependency:3.6.0:analyze (default-cli) < test-compile @
spark-common-utils_2.13 <<<
[INFO]
[INFO]
[INFO] --- dependency:3.6.0:analyze (default-cli) @ spark-common-utils_2.13 ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.fasterxml.jackson.core:jackson-annotations:jar:2.16.1:compile
[WARNING] org.apache.commons:commons-lang3:jar:3.14.0:compile
[WARNING] com.fasterxml.jackson.core:jackson-core:jar:2.16.1:compile
[WARNING] org.scala-lang:scala-library:jar:2.13.12:compile
[WARNING] org.scalatest:scalatest-funsuite_2.13:jar:3.2.17:test
[WARNING] org.scalactic:scalactic_2.13:jar:3.2.17:test
[WARNING] org.scalatest:scalatest-compatible:jar:3.2.17:test
[WARNING] org.scalatest:scalatest-core_2.13:jar:3.2.17:test
[WARNING] Unused declared dependencies found:
[WARNING]
com.fasterxml.jackson.module:jackson-module-scala_2.13:jar:2.16.1:compile
[WARNING] oro:oro:jar:2.0.8:compile
[WARNING] org.slf4j:jul-to-slf4j:jar:2.0.11:compile
[WARNING] org.slf4j:jcl-over-slf4j:jar:2.0.11:compile
[WARNING] org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.22.1:compile
[WARNING] org.apache.logging.log4j:log4j-1.2-api:jar:2.22.1:compile
[WARNING] org.spark-project.spark:unused:jar:1.0.0:compile
[WARNING] org.scalatest:scalatest_2.13:jar:3.2.17:test
[WARNING] org.scalatestplus:scalacheck-1-17_2.13:jar:3.2.17.0:test
[WARNING] org.scalatestplus:mockito-4-11_2.13:jar:3.2.17.0:test
[WARNING] org.scalatestplus:selenium-4-12_2.13:jar:3.2.17.0:test
[WARNING] org.junit.jupiter:junit-jupiter:jar:5.9.3:test
[WARNING] net.aichler:jupiter-interface:jar:0.11.1:test
[WARNING] Non-test scoped test only dependencies found:
[WARNING] commons-io:commons-io:jar:2.15.1:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.474 s (Wall Clock)
[INFO] Finished at: 2024-02-14T22:44:45+08:00
[INFO] ------------------------------------------------------------------------
{code}
> Fix `common-utils` module to have explicit `commons-lang3` dependency
> ---------------------------------------------------------------------
>
> Key: SPARK-47042
> URL: https://issues.apache.org/jira/browse/SPARK-47042
> Project: Spark
> Issue Type: Sub-task
> Components: Build, Tests
> Affects Versions: 4.0.0
> Reporter: William Wong
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 4.0.0
>
>
> Following scala code depends on `commons-lang3` explicitly. However, the
> common-utils modules missing related dependency.
> {code:java}
> ~/dev/sources/spark/common/utils$ grep -R lang3 * | grep import
> src/main/scala/org/apache/spark/util/MavenUtils.scala:import
> org.apache.commons.lang3.StringUtils
> src/main/scala/org/apache/spark/util/ClosureCleaner.scala:import
> org.apache.commons.lang3.ClassUtils
> src/main/java/org/apache/spark/network/util/JavaUtils.java:import
> org.apache.commons.lang3.SystemUtils; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
