Rajkumar created SPARK-19983:
--------------------------------
Summary: Getting ValidationFailureSemanticException on 'INSERT
OVEWRITE'
Key: SPARK-19983
URL: https://issues.apache.org/jira/browse/SPARK-19983
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.1.0
Reporter: Rajkumar
Priority: Blocker
Hi, I am creating a DataFrame and registering that DataFrame as temp table
using df.createOrReplaceTempView('mytable'). After that I try to write the
content from 'mytable' into Hive table(It has partition) using the following
query
insert overwrite table
myhivedb.myhivetable
partition(testdate) // ( 1) : Note here : I have a partition named 'testdate'
select
Field1,
Field2,
...
TestDate //(2) : Note here : I have a field named 'TestDate' ; Both (1) & (2)
have the same name
from
mytable
when I execute this query, I am getting the following error
Exception in thread "main"
org.apache.hadoop.hive.ql.metadata.Table$ValidationFailureSemanticException:
Partition spec {testdate=, TestDate=2013-01-01}
Looks like I am getting this error because of the same field names ; ie
testdate(the partition in Hive) & TestDate (The field in temp table 'mytable')
Whereas if my fieldname 'TestDate' is different, the query executes
successuflly. Example...
insert overwrite table
myhivedb.myhivetable
partition(testdate)
select
Field1,
Field2,
...
myDate //Note here : The field name is 'myDate' & not 'TestDate'
from
mytable
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
