Ethan Guo created HUDI-3880:
-------------------------------
Summary: Check file listing for common partitions when partition
list mismatches in the validation
Key: HUDI-3880
URL: https://issues.apache.org/jira/browse/HUDI-3880
Project: Apache Hudi
Issue Type: Improvement
Reporter: Ethan Guo
Assignee: Yue Zhang
Fix For: 0.12.0
{code:java}
org.apache.hudi.exception.HoodieValidationException: Compare Partitions Failed!
AllPartitionPathsFromFS : [2021/12/1, 2022/1/24, 2022/1/25, 2022/1/26,
2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2] and
allPartitionPathsMeta : [2021/12/1, 2021/12/2, 2022/1/24, 2022/1/25, 2022/1/26,
2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2]
{code}
When the partition listing is different between FS and metadata table, we can
still check the file listing for common partitions and print the result. For
the partition that only appears in one side, we can check if it is empty or
not. If it is, we can mark it as matching; otherwise, we throw an error for
the partition.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
