Github user zhzhan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9553#discussion_r44243045
--- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/Main.scala
---
@@ -78,16 +79,21 @@ object Main extends Logging {
}
sparkContext = new SparkContext(conf)
logInfo("Created spark context..")
+ sqlContextWithHive = conf.getBoolean("spark.sql.hive.context", true)
sparkContext
}
def createSQLContext(): SQLContext = {
- val name = "org.apache.spark.sql.hive.HiveContext"
+ val name = sqlContextWithHive match {
+ case true => "org.apache.spark.sql.hive.HiveContext"
+ case false => "org.apache.spark.sql.SQLContext"
+ }
+
val loader = Utils.getContextOrSparkClassLoader
try {
sqlContext =
loader.loadClass(name).getConstructor(classOf[SparkContext])
.newInstance(sparkContext).asInstanceOf[SQLContext]
- logInfo("Created sql context (with Hive support)..")
+ logInfo("Created sql context with " + name)
} catch {
case _: java.lang.ClassNotFoundException | _:
java.lang.NoClassDefFoundError =>
sqlContext = new SQLContext(sparkContext)
--- End diff --
I don't know how to fix this, but the logic seems to be weird if SQLContext
is used, then in the exception handler the context is created again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
