sarutak opened a new pull request, #55645: URL: https://github.com/apache/spark/pull/55645
### What changes were proposed in this pull request? This PR excludes old `junit:junit` from the dependency tree for sbt build. This is also another solution for SPARK-56478 (#55358). ### Why are the changes needed? Currently, we use `org.junit.jupiter:junit-jupiter` rather than `junit:junit`, and `junit:junit` is explicitly excluded in `pom.xml`. https://github.com/apache/spark/blob/8e37824402531f82ad1dfef415b5c29b478df760/pom.xml#L1519 But the dependency tree for sbt build still includes `junit:junit`. ``` $ build/sbt Test/dependencyTree ... [info] | +-org.apache.parquet:parquet-column:1.17.0 [info] | | +-com.carrotsearch:junit-benchmarks:0.7.2 [info] | | +-junit:junit:4.13.2 [info] | | | +-org.hamcrest:hamcrest-core:1.3 [info] | | | [info] | | +-org.apache.commons:commons-lang3:3.17.0 (evicted by: 3.20.0) [info] | | +-org.apache.commons:commons-lang3:3.20.0 [info] | | +-org.apache.parquet:parquet-common:1.17.0 [info] | | | +-junit:junit:4.13.2 [info] | | | | +-org.hamcrest:hamcrest-core:1.3 ``` `junit:junit` also depends on `org.hamcrest:hamcrest-core:1.3`, which is causes an issue. If, we build spark using Maven with `-Psparkr`, the pom file for `hamcrest-core:1.3` will be downloaded but the corresponding jar file will not. ``` $ build/mvn -DskipTests -Psparkr package $ ls ~/.m2/repository/org/hamcrest/hamcrest-core/1.3/ _remote.repositories hamcrest-core-1.3.pom hamcrest-core-1.3.pom.lastUpdated hamcrest-core-1.3.pom.sha1 ``` If we then build Spark using sbt in this situation, it will fail. ``` $ build/sbt package ... $ build/sbt package ... [error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts: [error] file:/home/release/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar: not found: /home/release/. m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar ``` For the same reason, building Scala/Java API document will fail. ``` $ cd docs $ SKIP_RDOC=1 SKIP_PYTHONDOC=1 SKIP_SQLDOC=1 bundler exec jekyll build ... [error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts: [error] file:/home/release/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar: not found: /home/release/. m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Confirmed both of the following commands succeeded ``` $ build/mvn -DskipTests -Psparkr package $ build/sbt package $ cd docs && SKIP_RDOC=1 SKIP_PYTHONDOC=1 SKIP_SQLDOC=1 bundler exec jekyll build ``` ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
