Gengliang Wang created SPARK-26230:
--------------------------------------
Summary: FileIndex: if case sensitive, validate partitions with
original column names
Key: SPARK-26230
URL: https://issues.apache.org/jira/browse/SPARK-26230
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0
Reporter: Gengliang Wang
Partition column name is required to be unique under the same directory. The
following paths are invalid partitioned directory:
```
hdfs://host:9000/path/a=1
hdfs://host:9000/path/b=2
```
If case sensitive, the following paths should be invalid too:
```
hdfs://host:9000/path/a=1
hdfs://host:9000/path/A=2
```
Since column 'a' and 'A' are different, and it is wrong to use either one as
the column name in partition schema.
Also, there is a `TODO` in the code. This PR is to resolve the problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
