yhuai commented on a change in pull request #27041: [SPARK-25061][SQL] 
Precedence for ThriftServer hiveconf commandline parameter
URL: https://github.com/apache/spark/pull/27041#discussion_r366528640
 
 

 ##########
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
 ##########
 @@ -177,8 +177,12 @@ private[hive] class HiveClientImpl(
     // has hive-site.xml. So, HiveConf will use that to override its default 
values.
     // 2: we set all spark confs to this hiveConf.
     // 3: we set all entries in config to this hiveConf.
+
+    // not to lose command line overwritten properties
+    // make a copy overridden props so that it can be reinserted finally
+    val overriddenHiveProps = HiveConf.getConfSystemProperties.asScala
     val confMap = (hadoopConf.iterator().asScala.map(kv => kv.getKey -> 
kv.getValue) ++
-      sparkConf.getAll.toMap ++ extraConfig).toMap
+      sparkConf.getAll.toMap ++ overriddenHiveProps ++ extraConfig).toMap
 
 Review comment:
   Seems we should update 
https://github.com/apache/spark/pull/27041/files#diff-6fd847124f8eae45ba2de1cf7d6296feR170-R179
 and also explain why extraConfig is at the end.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to