Kris Geusebroek created SPARK-24965:
---------------------------------------
Summary: Spark SQL fails when reading a partitioned hive table
with different formats per partition
Key: SPARK-24965
URL: https://issues.apache.org/jira/browse/SPARK-24965
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.1
Reporter: Kris Geusebroek
When a hive parquet partitioned table contains a partition with a different
format (avro for example) the select * fails with a read exception (avro file
is not a parquet file)
Selecting in hive acts as expected.
To support this a new sql syntax needed to be supported also:
* ALTER TABLE <tbl> <partiitonspec?> SET FILEFORMAT <format>
This is included in the same PR since the unittest needs this to setup the
testdata.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
