[
https://issues.apache.org/jira/browse/HIVE-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita resolved HIVE-25453.
-------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Committed to master. Thanks for the review [~pvary]!
> Add LLAP IO support for Iceberg ORC tables
> ------------------------------------------
>
> Key: HIVE-25453
> URL: https://issues.apache.org/jira/browse/HIVE-25453
> Project: Hive
> Issue Type: New Feature
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Adding support for reading Iceberg ORC tables via LLAP..
> The easy part is swapping out the plain simple VectorizedOrcRecordReader to
> LlapRecordReader.
> The hard part is maintaining correctness even after a series of schema
> changes that are normally allowed to Iceberg/ORC, but were not for simple ORC
> or therefore for LLAP. To make it all work, LLAP had to be made to support a
> broader schema evolution.
> Before this change LLAP made the simple assumption that the reader and file
> schemas match all columns, now separate physical and logical read schemas and
> corresponding include lists are used instead. Also added
> logicalOrderedColumnIds here, which holds indices from the reader schema, but
> in file schema order - a necessary tool for mapping the results produced by
> LLAP, as LLAP always reads columns in the order as they are written out in
> the file.
> Also added a new CLI driver class for testing the cached reads from
> Iceberg/ORC tables via LLAP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
