EnricoMi opened a new pull request, #1523: URL: https://github.com/apache/incubator-uniffle/pull/1523
### What changes were proposed in this pull request? Make missing `shuffle-server` dependencies explicit in `server/pom.xml`. ### Why are the changes needed? The `shuffle-server` module has dependencies that are not explicitly stated in the `pom.xml`, but transitively available. Notably, these are `commons-collections` and `hadoop-common`: https://github.com/apache/incubator-uniffle/blob/866f5ff121948e2144729300a0f5cb03510db29d/server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java#L27-L29 Run `./mvnw dependency:tree -pl org.apache.uniffle:shuffle-server -am` to see the dependency tree: ``` [INFO] -----------------< org.apache.uniffle:shuffle-server >------------------ [INFO] Building Apache Uniffle Server 0.9.0-SNAPSHOT [6/6] [INFO] from server/pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] … [INFO] --- dependency:2.10:tree (default-cli) @ shuffle-server --- [INFO] org.apache.uniffle:shuffle-server:jar:0.9.0-SNAPSHOT [INFO] … [INFO] +- org.apache.hadoop:hadoop-minicluster:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-common:test-jar:tests:2.8.5:test [INFO] … [INFO] | | +- commons-collections:commons-collections:jar:3.2.2:provided [INFO] … [INFO] | +- org.apache.hadoop:hadoop-common:jar:2.8.5:provided ``` Both dependencies are available through `org.apache.hadoop:hadoop-minicluster` as `provided`. However, with `-Ptez`, the dependency tree changes in such a way that those transitive dependency become available only in `test` scope, breaking compilation of the `shuffle-server` module: ``` ./mvnw dependency:tree -pl org.apache.uniffle:shuffle-server -am -Ptez ``` ``` [INFO] -----------------< org.apache.uniffle:shuffle-server >------------------ [INFO] Building Apache Uniffle Server 0.9.0-SNAPSHOT [6/6] [INFO] from server/pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] … [INFO] --- dependency:2.10:tree (default-cli) @ shuffle-server --- [INFO] org.apache.uniffle:shuffle-server:jar:0.9.0-SNAPSHOT [INFO] … [INFO] +- org.apache.hadoop:hadoop-minicluster:jar:2.8.5:test [INFO] … [INFO] | | +- commons-collections:commons-collections:jar:3.2.2:compile [INFO] … [INFO] | +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile ``` The required dependencies are now in `compile` scope under a `test` scope dependency. The `mvnw` command line build tool seems robust against this, which is why CI works just fine. But IntelliJ IDE seems to handle those dependency trees literally and compilation breaks. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
