[
https://issues.apache.org/jira/browse/HUDI-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HUDI-2443:
-------------------------------------
Status: Patch Available (was: In Progress)
> KVComparator in HFile for metadata table is tied to HBase version and shading
> -----------------------------------------------------------------------------
>
> Key: HUDI-2443
> URL: https://issues.apache.org/jira/browse/HUDI-2443
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Ryan Pifer
> Assignee: Manoj Govindassamy
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> There is no guarantee of compatibility between HFile reading/writing from
> different versions of ** HBase. For instance, HBase version 2.2.6 modified
> the comparator class stored in HFile metadata to
> {{org.apache.hadoop.hbase.KeyValue.KVComparator}} and
> {{org.apache.hadoop.hbase.KeyValue$KeyComparator}} no longer exists which may
> create issues when reading HFiles created by earlier versions of Hudi. And
> this comparator class is already marked as deprecated for
> {{CellComparatorImpl.}}
> {{This also creates issues when shading HBase dependencies under Hudi
> namespace}}
> {{Can we refactor HoodieKVComparator which is used in bootstrap index and
> have it applied for all use of HFile in Hudi. }}
> {{[https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java#L580-L586]}}
>
> There will need to be some logic as well for backward compatibility when
> reading metadata which was written from earlier version of Hudi as well. We
> can maybe enforce that user must perform an upgrade step beforehand which
> re-writes metadata base file
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
