AngersZhuuuu commented on pull request #34830: URL: https://github.com/apache/spark/pull/34830#issuecomment-989444661
> Could you describe a bit more on the use case @AngersZhuuuu ? This code path is only invoked when `spark.sql.hive.metastore.jars` is `maven` which is very slow comparing to other options. I know, but for such case that we use a spark built with hadoop-3.3 and hive-2.3 and set `spark.sql.hive.metastore.jars` to `maven`, it should download hadoop3.3 related jar, if it failed to download, it will fallback to hadoop-2.7.4 for current code path. Then will have exception I show in the pr desc. In our case, we change the buildin hadoop version to our internal version and have a problem to download it. it fallback to hadoop-2.7.4. It's not correct, it should fallback to hadoop-3.3.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
