Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5876#discussion_r29883543
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala ---
@@ -93,9 +99,129 @@ class HiveContext(sc: SparkContext) extends
SQLContext(sc) {
protected[sql] def convertCTAS: Boolean =
getConf("spark.sql.hive.convertCTAS", "false").toBoolean
+ /**
+ * The version of the hive client that will be used to communicate with
the metastore. Note that
+ * this does not necessarily need to be the same version of Hive that is
used internally by
+ * Spark SQL for execution.
+ */
+ protected[hive] def hiveMetastoreVersion: String =
+ getConf(HIVE_METASTORE_VERSION, "0.13.1")
+
+ /**
+ * The location of the jars that should be used to instantiate the
HiveMetastoreClient. This
+ * property can be one of three options:
+ * - a colon-separated list of jar files or directories for hive and
hadoop.
+ * - builtin - attempt to discover the jars that were used to load
Spark SQL and use those. This
+ * option is only valid when using the execution version of
Hive.
+ * - maven - download the correct version of hive on demand from maven.
+ */
+ protected[hive] def hiveMetastoreJars: String =
+ getConf(HIVE_METASTORE_JARS, "builtin")
+
@transient
protected[sql] lazy val substitutor = new VariableSubstitution()
+
+ /** A local instance of hive that is only used for execution. */
+ protected[hive] lazy val localMetastore = {
+ val temp = Utils.createTempDir()
+ temp.delete()
+ temp
+ }
+
+ @transient
+ protected[hive] lazy val executionConf = new HiveConf()
+ executionConf.set(
+ "javax.jdo.option.ConnectionURL",
s"jdbc:derby:;databaseName=$localMetastore;create=true")
--- End diff --
Correct, but some operations do instantiate the client so we need to point
it at a dummy database otherwise in many cases derby will complain that more
that one processes is trying to open the database.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
