[
https://issues.apache.org/jira/browse/HUDI-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Li resolved HUDI-1611.
---------------------------
Resolution: Resolved
> Allow directories to be filtered during the bootstrap of the metadata table
> ---------------------------------------------------------------------------
>
> Key: HUDI-1611
> URL: https://issues.apache.org/jira/browse/HUDI-1611
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Prashant Wason
> Assignee: Prashant Wason
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.8.0
>
>
> During the bootstrap of the Metadata Table, all the directories which contain
> the partition metadata directory are assumed to be partitions and are added
> to the metadata table.
> In our HDFS clusters, we have directories like .backup, .temp which are used
> by various teams for non-hoodie purposes (e.g. .backup may be keeping a
> snapshot of the dataset). During bootstrap, Metadata Table ends up containing
> all those paths also as partitions.
> In this patch, I would like to introduce a configuration for
> HoodieMetadataConfig to filter out some directories based on a regular
> expression string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
