Danny Chen created HUDI-9318:
--------------------------------
Summary: Refactor the log records presentation in
FileGroupRecordBuffer
Key: HUDI-9318
URL: https://issues.apache.org/jira/browse/HUDI-9318
Project: Apache Hudi
Issue Type: Improvement
Components: core
Reporter: Danny Chen
Assignee: Timothy Brown
Fix For: 1.1.0
The records cache in FileGroupRecordBuffer is map, the map key is the record
key, the map value is a pair:
{code:java}
Pair<Option<T>, Map<String, Object>>{code}
the pair left is the actual option of HoodieRecord, an empty options represents
a delete;
the pair right is the record metadata, such as record key, orderingValue, and
avro schema.
Let's revise the metadata part:
1. add a new serializable Java class MergingItem to holds the hoodie record;
2. the record key and orderingValue should be fetched through HoodieRecord
APIs, we need to check all the hoodie record types to ensure this is efficient,
if not try to fix it as efficient, and if it is hard to do that for all kinds
of record type, let's put a duplicate in the MergingItem class;
3. does not emptify the delete record as an Option.empty, set up the `isDelete`
correctly in HoodieRecord itself or put a duplicate in MergingItem class;
4. add the local schema id in the MergingItem class;
5. drop out the metadata map which is very confusing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
