ajithme commented on a change in pull request #27041: [SPARK-25061][SQL]
Precedence for ThriftServer hiveconf commandline parameter
URL: https://github.com/apache/spark/pull/27041#discussion_r380731555
##########
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -176,9 +176,18 @@ private[hive] class HiveClientImpl(
// is not set to builtin. When spark.sql.hive.metastore.jars is builtin,
the classpath
// has hive-site.xml. So, HiveConf will use that to override its default
values.
// 2: we set all spark confs to this hiveConf.
- // 3: we set all entries in config to this hiveConf.
+ // 3: we take the conf passed as --hiveconf which would be set as system
properties
+ // by
org.apache.hive.service.server.HiveServer2.ServerOptionsProcessor.parse in
+ // org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main.
+ // 4: we set all entries in extraConfig to this hiveConf which have the
highest precedence.
+ // To summarize, the order of precedence will be
+ // hadoopConf < sparkConf < overrideProps < extraConfig
+
+ // not to lose command line overwritten properties
+ // make a copy overridden props so that it can be reinserted finally
+ val overriddenHiveProps = HiveConf.getConfSystemProperties.asScala
Review comment:
@cloud-fan updated. is the comment adequate now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
