[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593152#comment-16593152
]
yucai commented on SPARK-25175:
-------------------------------
I pinged [~seancxmao] offline, he will give more details.
> Case-insensitive field resolution when reading from ORC
> -------------------------------------------------------
>
> Key: SPARK-25175
> URL: https://issues.apache.org/jira/browse/SPARK-25175
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Chenxiao Mao
> Priority: Major
>
> SPARK-25132 adds support for case-insensitive field resolution when reading
> from Parquet files. We found ORC files have similar issues. Since Spark has 2
> OrcFileFormat, we should add support for both.
> * Since SPARK-2883, Spark supports ORC inside sql/hive module with Hive
> dependency. This hive OrcFileFormat doesn't support case-insensitive field
> resolution at all.
> * SPARK-20682 adds a new ORC data source inside sql/core. This native
> OrcFileFormat supports case-insensitive field resolution, however it cannot
> handle duplicate fields.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
