[
https://issues.apache.org/jira/browse/HUDI-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133871#comment-17133871
]
sivabalan narayanan commented on HUDI-635:
------------------------------------------
[~vinoth]: Have a follow up question on fixing keysToNewRecords to
keysToNewRecordPayloads. One of the constructors of HoodieMergeHandle directly
sends the keyToNewRecords.
```
/**
* Called by compactor code path.
*/
public HoodieMergeHandle(HoodieWriteConfig config, String instantTime,
HoodieTable<T> hoodieTable,
Map<String, T> keyToNewRecordPayloads, String partitionPath, String fileId,
HoodieBaseFile dataFileToBeMerged, SparkTaskContextSupplier
sparkTaskContextSupplier) {
```
I triaged the callstack and the argument is coming from
HoodieMergedLogRecordScanner.getRecords(). So can't really change this. Should
I iterate this within HoodieMergeHandle and convert to keysToNewRecordPayloads?
But that might be an additional computation compared to what we have now.
> MergeHandle's DiskBasedMap entries can be thinner
> -------------------------------------------------
>
> Key: HUDI-635
> URL: https://issues.apache.org/jira/browse/HUDI-635
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Performance, Writer Core
> Reporter: Vinoth Chandar
> Assignee: sivabalan narayanan
> Priority: Blocker
> Labels: help-requested
> Fix For: 0.6.0
>
>
> Instead of <Key, HoodieRecord>, we can just track <Key, Payload> ... Helps
> with use-cases like HUDI-625
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
