Zihan Li created GOBBLIN-877:

             Summary: Add column metadata for partition for inline hive 
                 Key: GOBBLIN-877
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-877
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Zihan Li

Previously, we remove the schema.literal for partition.  Because Avro schemas 
should _only_ be defined at the table level. Hive overrides table properties if 
the same property is defined on the partition. Defining them at the partition 
level may lead to partitions with inconsistent schemas. And because column 
metadata is calculated from schema.literal, so we remove the column metadata as 

Then we encounter a problem that presto cannot read data from orc file. Because 
ORC (and other Hive serdes) need metadata in the partitions so that coercion 
can be done between a partition schema and the table schema.

So we need to treat Avro and other formate separately to make sure hive 
registration works well so that user can read right data from Presto.

This message was sent by Atlassian Jira

Reply via email to