[
https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14983278#comment-14983278
]
holdenk commented on SPARK-8586:
--------------------------------
I've got a related fix in https://issues.apache.org/jira/browse/SPARK-11421 we
could maybe try and use it here (if that gets merged in)
> SQL add jar command does not work well with Scala REPL
> ------------------------------------------------------
>
> Key: SPARK-8586
> URL: https://issues.apache.org/jira/browse/SPARK-8586
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.4.0
> Reporter: Yin Huai
> Assignee: Yin Huai
> Priority: Critical
>
> Seems SparkIMain always resets the context class loader in {{loadAndRunReq}}.
> So, SerDe added through add jar command may not be loaded in the context
> class loader when we lookup the table.
> For example, the following code will fail when we try to show the table.
> {code}
> hive.sql("add jar sql/hive/src/test/resources/hive-hcatalog-core-0.13.1.jar")
> hive.sql("drop table if exists jsonTable")
> hive.sql("CREATE TABLE jsonTable(key int, val string) ROW FORMAT SERDE
> 'org.apache.hive.hcatalog.data.JsonSerDe'")
> hive.createDataFrame((1 to 100).map(i => (i, s"str$i"))).toDF("key",
> "val").insertInto("jsonTable")
> hive.table("jsonTable").show
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
