[
https://issues.apache.org/jira/browse/ORC-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15759643#comment-15759643
]
Lefty Leverenz commented on ORC-120:
------------------------------------
This will need documentation, as discussed in some Dec. 12 messages on the
dev@orc email thread, archived here:
*
https://mail-archives.apache.org/mod_mbox/orc-dev/201612.mbox/%[email protected]%3e
It's strange that the messages don't appear here in the JIRA.
Synopsis:
Dain Sundstrom -- Is "ORC's schema evolution uses the column names when they
are available” documented somewhere?
Owen O'Malley -- No, unfortunately, but it needs to be. The basic rules from
SchemaEvolution.java look like:
structs (including the top row):
if field names are available (post HIVE-4243), use name matching
otherwise use positional matching
lists, maps, unions:
children must match
Many primitives can convert to each other, but this list needs to be cleaned up:
boolean, byte, short, int, long, float, double, decimal -> boolean, byte,
short, int, long, float, double, decimal, string, char, varchar, timestamp
string, char, varchar -> all
timestamp -> boolean, byte, short, int, long, float, double, decimal,
string, char, varchar, date
date -> string, char, varchar, timestamp
binary -> string, char, varchar, date
Dain Sundstrom -- So, rename column is not expected to work anymore?
Owen O'Malley -- ORC-120 will add an option to force positional mapping.
Dain Sundstrom -- Oh, I see this is an ORC feature like the Parquet schema
evolution stuff. We implemented support for ordering by the top level struct
names in Presto a while back.
> Create a backwards compatibility mode of ignoring names for evolution
> ---------------------------------------------------------------------
>
> Key: ORC-120
> URL: https://issues.apache.org/jira/browse/ORC-120
> Project: Orc
> Issue Type: Task
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 1.3.0
>
>
> ORC's schema evolution uses the column names when they are available. Hive
> 2.1 uses a positional schema, so ORC should support a backward compatibility
> mode for Hive users during the transition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
