[
https://issues.apache.org/jira/browse/HUDI-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801635#comment-17801635
]
vmaster.cc commented on HUDI-7051:
----------------------------------
[~shivnarayan] thanks for your reply, as you say, the class of
DayBasedCompactionStrategy has been fixed by issue HUDI-6975, but problem still
exists in follow class:
#
org.apache.hudi.table.action.compact.strategy.BoundedPartitionAwareCompactionStrategy
#
org.apache.hudi.table.action.compact.strategy.UnBoundedPartitionAwareCompactionStrategy
if we have a hive partition like this: dllr_date=2023-10-10 ,the replace
operation will result in an error, final get 'dllr_date=2023/10/10'
{code:java}
List<String> allPartitionPaths =
partitionPaths.stream().map(partition -> partition.replace("/",
"-")).sorted(Comparator.reverseOrder())
.map(partitionPath -> partitionPath.replace("-",
"/")).collect(Collectors.toList()); {code}
in other words, there has second replace operation, but the first will has no
effect, only the second replace works, as this point, there is no converted
back.
> Incorrect replace operation in compaction strategy filter
> ---------------------------------------------------------
>
> Key: HUDI-7051
> URL: https://issues.apache.org/jira/browse/HUDI-7051
> Project: Apache Hudi
> Issue Type: Bug
> Components: compaction
> Reporter: vmaster.cc
> Priority: Major
> Attachments: image-2023-11-08-16-01-46-166.png,
> image-2023-11-08-16-02-39-291.png
>
>
> There are some incorrect replace operation to sort all partition paths.
> {code:java}
> return allPartitionPaths.stream().map(partition -> partition.replace("/",
> "-"))
> .sorted(Comparator.reverseOrder()).map(partitionPath ->
> partitionPath.replace("-", "/")) {code}
> the hive partition before replace is dllr_date=2023-10-10, then after will
> convert to dllr_date=2023/10/10, this is an incorrect partition.
> # org.apache.hudi.table.action.compact.strategy.DayBasedCompactionStrategy
> #
> org.apache.hudi.table.action.compact.strategy.BoundedPartitionAwareCompactionStrategy
> #
> org.apache.hudi.table.action.compact.strategy.UnBoundedPartitionAwareCompactionStrategy
> !image-2023-11-08-16-02-39-291.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
