Yin Huai created SPARK-6023:
-------------------------------
Summary: ParquetConversions fails to replace the destination
MetastoreRelation to ParquetRelation2
Key: SPARK-6023
URL: https://issues.apache.org/jira/browse/SPARK-6023
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Yin Huai
Priority: Blocker
{code}
import sqlContext._
sql("drop table if exists test")
val df1 = sqlContext.jsonRDD(sc.parallelize((1 to 10).map(i =>
s"""{"a":$i}""")))
df1.registerTempTable("jt")
sql("create table test (a bigint) stored as parquet ")
sql("explain insert into table test select a from jt").collect.foreach(println)
{code}
The plan will be
{code}
[== Physical Plan ==]
[InsertIntoHiveTable (MetastoreRelation default, test, None), Map(), false]
[ PhysicalRDD [a#34L], MapPartitionsRDD[17] at map at JsonRDD.scala:41]
{code}
However, the write path should be converted to our own data source path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
