sunchao commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-735980995
> Or maybe we could try to shade the hadoop-aws jar in the spark-hadoop-cloud_2.12 module itself so that it invokes the shaded API from hadoop-common side. This won't work if Spark users decide to use their own Hadoop jars (via hadoop-provided) so we may have to make the hadoop-aws a compile scope dependency. I created a [WIP PR](https://github.com/apache/spark/pull/30556) to demonstrate how this works. Feel free to comment on it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
