Suresh Thalamati created SPARK-15538:
----------------------------------------
Summary: Truncate table does not work on data source table , and
does not raise error either.
Key: SPARK-15538
URL: https://issues.apache.org/jira/browse/SPARK-15538
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Suresh Thalamati
Priority: Minor
Truncate table does not seems to work on data source table. It returns success
without any error , but table is not truncated.
Repro:
{code}
val df = Seq((1 , "john", "CA") ,(2,"Mike", "NY"), (3, "Robert",
"CA")).toDF("id", "name", "state")
df.write.format("parquet").partitionBy("state").saveAsTable("emp")
scala> sql("truncate table emp")
res8: org.apache.spark.sql.DataFrame = []
scala> sql("select * from emp").show ;
+---+------+-----+
| id| name|state|
+---+------+-----+
| 3|Robert| CA|
| 1| john| CA|
| 2| Mike| NY|
+---+------+-----+
{code}
The select should have returned no results.
By scanning through the code I found some of the other DDL commands like
LOAD DATA , and SHOW PARTITIONS are not allowed for data source table and they
raise error.
It Might be good to throw error until the truncate table works with data
source table also.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
