yaooqinn commented on a change in pull request #30978:
URL: https://github.com/apache/spark/pull/30978#discussion_r550197312
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -58,23 +58,6 @@ private[sql] class SharedState(
// both spark conf and hadoop conf avoiding be affected by any
SparkSession level options
SharedState.loadHiveConfFile(
sparkContext.conf, sparkContext.hadoopConfiguration, initialConfigs)
- val confClone = sparkContext.conf.clone()
- val hadoopConfClone = new Configuration(sparkContext.hadoopConfiguration)
- // If `SparkSession` is instantiated using an existing `SparkContext`
instance and no existing
- // `SharedState`, all `SparkSession` level configurations have higher
priority to generate a
- // `SharedState` instance. This will be done only once then shared across
`SparkSession`s
- initialConfigs.foreach {
- case (k, _) if k == "hive.metastore.warehouse.dir" || k ==
WAREHOUSE_PATH.key =>
- logWarning(s"Not allowing to set ${WAREHOUSE_PATH.key} or
hive.metastore.warehouse.dir " +
Review comment:
The warning message here is incorrect, the `spark.sql.warehouse.dir`
should already be used in `loadHiveConfFile`.
So in this pr, we will only print this warning when hit
`hive.metastore.warehouse.dir` in `initialConfigs`. It is moved to
.https://github.com/apache/spark/pull/30978/files#diff-45744d86661942d414a6e12f72e1d24f62b1db1ce1a5a21a60dfb52abf7611b0R235
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
