[
https://issues.apache.org/jira/browse/SPARK-26234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joby Joje updated SPARK-26234:
------------------------------
Description:
While trying to OVERWRITE the Hive table with specific columns from
Spark(Pyspark) using a dataframe getting the below error
{quote}pyspark.sql.utils.ParseException: u"\nmismatched input 'col1' expecting
Unknown macro: \{'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP',
'REDUCE'}
(line 1, pos 36)\n\n== SQL ==\ninsert into table DB.TableName (Col1, Col2,
Col3) select Col1, Col2, Col3 FROM
dataframe\n------------------------------------^^^\n"
{quote}
{{sparkSession.sql("insert into table DB.TableName (Col1, Col2, Col3) select
Col1, Col2, Col3 FROM dataframe")}}
{{But on trying the same via _Hive Terminal_ goes through fine.}}
Please check the below link to get more info on the same.
[https://stackoverflow.com/questions/53517671/column-list-specification-in-insert-overwrite-statement]
was:
While trying to OVERWRITE the Hive table with specific columns from
Spark(Pyspark) using a dataframe getting the below error
{quote}pyspark.sql.utils.ParseException: u"\nmismatched input 'col1' expecting
Unknown macro: \{'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP',
'REDUCE'}
(line 1, pos 36)\n\n== SQL ==\ninsert table DB.TableName (Col1, Col2, Col3)
select Col1, Col2, Col3 FROM
dataframe\n------------------------------------^^^\n"
{quote}
{{sparkSession.sql("insert into table DB.TableName (Col1, Col2, Col3) select
Col1, Col2, Col3 FROM dataframe")}}
{{But on trying the same via _Hive Terminal_ goes through fine.}}
Please check the below link to get more info on the same.
[https://stackoverflow.com/questions/53517671/column-list-specification-in-insert-overwrite-statement]
> Column list specification in INSERT statement
> ---------------------------------------------
>
> Key: SPARK-26234
> URL: https://issues.apache.org/jira/browse/SPARK-26234
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.3.0
> Reporter: Joby Joje
> Priority: Major
>
> While trying to OVERWRITE the Hive table with specific columns from
> Spark(Pyspark) using a dataframe getting the below error
> {quote}pyspark.sql.utils.ParseException: u"\nmismatched input 'col1' expecting
> Unknown macro: \{'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP',
> 'REDUCE'}
> (line 1, pos 36)\n\n== SQL ==\ninsert into table DB.TableName (Col1, Col2,
> Col3) select Col1, Col2, Col3 FROM
> dataframe\n------------------------------------^^^\n"
> {quote}
> {{sparkSession.sql("insert into table DB.TableName (Col1, Col2, Col3) select
> Col1, Col2, Col3 FROM dataframe")}}
> {{But on trying the same via _Hive Terminal_ goes through fine.}}
> Please check the below link to get more info on the same.
> [https://stackoverflow.com/questions/53517671/column-list-specification-in-insert-overwrite-statement]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
