Cheng Lian created SPARK-14918:
----------------------------------
Summary: ExternalCatalog.TablePartitionSpec doesn't preserve
partition column order
Key: SPARK-14918
URL: https://issues.apache.org/jira/browse/SPARK-14918
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.0
Reporter: Cheng Lian
Assignee: Cheng Lian
Equivalent entity of {{ExternalCatalog.TablePartitionSpec}} in Hive is a
{{LinkedHashMap}} returned by {{Partition.getSpec()}}, which preserves
partition column order.
However, we are using a {{scala.immutable.Map}} to store the result, which no
longer preserves the original order. What makes it worse, Scala specializes
immutable maps with less than 5 elements. And these specialized versions do
preserve order, thus hides this issue in test cases since we never use more
than 4 partition columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
