Yin Huai created SPARK-8586:
-------------------------------
Summary: SQL add jar command does not work well with Scala REPL
Key: SPARK-8586
URL: https://issues.apache.org/jira/browse/SPARK-8586
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.4.0
Reporter: Yin Huai
Assignee: Yin Huai
Priority: Critical
Seems SparkIMain always resets the context class loader in {{loadAndRunReq}}.
So, SerDe added through add jar command may not be loaded in the context class
loader when we lookup the table.
For example, the following code will fail when we try to show the table.
{code}
hive.sql("add jar sql/hive/src/test/resources/hive-hcatalog-core-0.13.1.jar")
hive.sql("drop table if exists jsonTable")
hive.sql("CREATE TABLE jsonTable(key int, val string) ROW FORMAT SERDE
'org.apache.hive.hcatalog.data.JsonSerDe'")
hive.createDataFrame((1 to 100).map(i => (i, s"str$i"))).toDF("key",
"val").insertInto("jsonTable")
hive.table("jsonTable").show
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
