kavn qin created SPARK-19878:
--------------------------------
Summary: Add hive configuration when initialize hive serde in
InsertIntoHiveTable.scala
Key: SPARK-19878
URL: https://issues.apache.org/jira/browse/SPARK-19878
Project: Spark
Issue Type: Wish
Components: SQL
Affects Versions: 2.0.0, 1.6.0, 1.5.0
Environment: Centos 6.5: Hadoop 2.6.0, Spark 1.5.0, Hive 1.1.0
Reporter: kavn qin
Priority: Minor
When case class InsertIntoHiveTable intializes a serde it explicitly passes
null for the Configuration in Spark 1.5.0:
[https://github.com/apache/spark/blob/v1.5.0/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala#L58]
While in Spark 2.0.0, the HiveWriterContainer intializes a serde it also just
passes null for the Configuration:
[https://github.com/apache/spark/blob/v2.0.0/sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveWriterContainers.scala#L161]
When we implement a hive serde, we want to use the hive configuration to get
some static and dynamic settings, we can not do it!
This patch just add the configuration when initialize hive serde.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
