GitHub user liutang123 opened a pull request:
https://github.com/apache/spark/pull/20846
[SPARK-5498][SQL][FOLLOW] add schema to table partition
## What changes were proposed in this pull request?
When query a orc table witch some partition schemas are different from
table schema, ClassCastException will occured.
reproduction:
`create table test_par(a string)
PARTITIONED BY (`b` bigint)
ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat';
ALTER TABLE test_par CHANGE a a bigint restrict; -- in hive
select * from test_par;`
## How was this patch tested?
manual test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liutang123/spark SPARK-5498
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20846.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20846
----
commit 2317bfdf18fc1a7b21cd43e0ec12f5e957fb1895
Author: liutang123 <liutang123@...>
Date: 2017-06-21T04:27:42Z
Merge pull request #1 from apache/master
20170521 pull request
commit 821b1f88e15bbe2bf7147f9cfa2e39ce7cb52b12
Author: liutang123 <liutang123@...>
Date: 2017-11-24T08:54:11Z
Merge branch 'master' of https://github.com/liutang123/spark
commit 1841f60861a96fb1508257c84e8703ca1ffb57de
Author: liutang123 <liutang123@...>
Date: 2017-11-24T08:54:59Z
Merge branch 'master' of https://github.com/apache/spark
commit 16f4a52aa556cdc5182570979bad9b4cf6f092d5
Author: liutang123 <liutang123@...>
Date: 2018-03-16T10:10:57Z
Merge branch 'master' of https://github.com/apache/spark
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
