sunchao commented on a change in pull request #34100:
URL: https://github.com/apache/spark/pull/34100#discussion_r715955928
##########
File path: pom.xml
##########
@@ -3273,7 +3273,7 @@
<curator.version>2.7.1</curator.version>
<commons-io.version>2.4</commons-io.version>
<hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact>
-
<hadoop-client-runtime.artifact>hadoop-client</hadoop-client-runtime.artifact>
+
<hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact>
Review comment:
Thanks for taking a look. Yes I think it's better to apply the same for
`hadoop-client-minicluster.artifact. Let me try that, and perhaps we won't need
the changes in YARN's pom.xml with this.
The side effect for this is seems to be that it affects the _distance_ of
these dependencies to the root module and thus may make a difference when maven
tries to resolve a dependency with multiple versions (see
[here](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html)
for reference). I was using `hadoop-common` (which carries lots of
dependencies) instead of `hadoop-yarn-api` and it was not able to compile.
Will update PR description and the comment in the above pom.xml.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
