[
https://issues.apache.org/jira/browse/SPARK-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013030#comment-15013030
]
pin_zhang commented on SPARK-11748:
-----------------------------------
Apache hive 0.14 has added Support for Parquet Column Rename
https://issues.apache.org/jira/browse/HIVE-6938
That doesn't work in spark hive
> Result is null after alter column name of table stored as Parquet
> ------------------------------------------------------------------
>
> Key: SPARK-11748
> URL: https://issues.apache.org/jira/browse/SPARK-11748
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.1
> Reporter: pin_zhang
>
> 1. Test with the following code
> hctx.sql(" create table " + table + " (id int, str string) STORED AS
> PARQUET ")
> val df = hctx.jsonFile("g:/vip.json")
> df.write.format("parquet").mode(SaveMode.Append).saveAsTable(table)
> hctx.sql(" select * from " + table).show()
> // alter table
> val alter = "alter table " + table + " CHANGE id i_d int "
> hctx.sql(alter)
>
> hctx.sql(" select * from " + table).show()
> 2. Result
> after change table column name, data in null for the changed column
> Result before alter table
> +---+---+
> | id|str|
> +---+---+
> | 1| s1|
> | 2| s2|
> +---+---+
> Result after alter table
> +----+---+
> | i_d|str|
> +----+---+
> |null| s1|
> |null| s2|
> +----+---+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
