[
https://issues.apache.org/jira/browse/HUDI-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kazdy updated HUDI-6232:
------------------------
Status: Patch Available (was: In Progress)
> Support skip archive for glue sync
> ----------------------------------
>
> Key: HUDI-6232
> URL: https://issues.apache.org/jira/browse/HUDI-6232
> Project: Apache Hudi
> Issue Type: Improvement
> Components: meta-sync
> Reporter: kazdy
> Assignee: kazdy
> Priority: Major
> Fix For: 0.14.0
>
>
> Glue has soft limit of 100k table versions. With every table update new
> version is created. In streaming scenarios we can hit 100k table versions
> quickly.
> Users should be able to skip table archival when using glue meta sync.
> It's been asked many times in Hudi Slack, how to reduce no of glue table
> versions created. There is conditional hive sync, but it does not solve the
> problem fully. With .skipArchive users can reduce no. of table versions
> created even more.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
