[jira] [Commented] (HUDI-8361) Revisit every fileSystemView.getLatestFileSlices in metadata writer/util and confirm if getLatestMergedFileSlices should be called

[sivabalan narayanan (Jira)]

    [ 
https://issues.apache.org/jira/browse/HUDI-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17890764#comment-17890764
 ] 

sivabalan narayanan commented on HUDI-8361:
-------------------------------------------

1. Functional and Sec index might need to be fixed. 
HoodieBackedTableMetadataWriter. 

 
{code:java}
private List<Pair<String, FileSlice>> getPartitionFileSlicePairs() throws 
IOException {
  try (HoodieMetadataFileSystemView fsView = getMetadataView()) {
    // Collect the list of latest file slices present in each partition
    List<String> partitions = metadata.getAllPartitionPaths();
    fsView.loadAllPartitions();
    List<Pair<String, FileSlice>> partitionFileSlicePairs = new ArrayList<>();
    partitions.forEach(partition -> 
fsView.getLatestFileSlices(partition).forEach(fs -> 
partitionFileSlicePairs.add(Pair.of(partition, fs))));
    return partitionFileSlicePairs;
  }
} {code}

fsView.getLatestFileSlices need to be replaced w/ 
fsView.getLatestMergedFileSlices

2. RLI initialization needs to be fixed 

initializeRecordIndexPartition 

3. HoodieBackedTableMetadataWriter.getFunctionalIndexUpdates -> 

HoodieTableMetadataUtil.getPartitionLatestFileSlicesIncludingInflight() 

 

 

 

> Revisit every fileSystemView.getLatestFileSlices in metadata writer/util and 
> confirm if getLatestMergedFileSlices should be called
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-8361
>                 URL: https://issues.apache.org/jira/browse/HUDI-8361
>             Project: Apache Hudi
>          Issue Type: Task
>            Reporter: Sagar Sumit
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>             Fix For: 1.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)