LuciferYang commented on code in PR #48015:
URL: https://github.com/apache/spark/pull/48015#discussion_r1749295915
##########
launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java:
##########
@@ -185,6 +188,13 @@ List<String> buildClassPath(String appClassPath) throws
IOException {
if (isRemote && "1".equals(getenv("SPARK_SCALA_SHELL")) &&
project.equals("sql/core")) {
continue;
}
+ // SPARK-49534: The assumption here is that if `spark-hive_xxx.jar`
is not in the classpath,
+ // then the `-Phive` profile was not used during package, and
therefore the Hive-related jars
+ // should also not be in the classpath. To avoid failure in loading
the SPI in `DataSourceRegister`
+ // under `sql/hive`, no longer prepend `sql/hive`.
+ if (!isSparkHiveJarAvailable && project.equals("sql/hive")) {
Review Comment:
Change to:
1. If `spark-hive_` does not exist in the classpath, do not prepend
`sql/hive`.
2. If `spark-hive_` or `spark-hive-thriftserver_` does not exist in the
classpath, do not prepend `sql/hive-thriftserver`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
