[
https://issues.apache.org/jira/browse/HUDI-308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-308:
--------------------------------
Labels: pull-request-available (was: )
> Avoid Renames for tracking state transitions of all actions on dataset
> ----------------------------------------------------------------------
>
> Key: HUDI-308
> URL: https://issues.apache.org/jira/browse/HUDI-308
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: Common Core
> Reporter: Balaji Varadarajan
> Assignee: Balaji Varadarajan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.1
>
> Attachments: IMG_0118.jpg
>
>
> Currently, We employ renames when transitioning states (REQUESTED, INFLIGHT,
> COMPLETED) of all actions in Hudi.
> The idea is to always create new files pertaining to each state of an action
> (commit, compaction, clean, ....) that is being performed and have the
> Timeline management resolve conflicts when loading them from .hoodie to
> folder. The Archiving logic will cleanup transient state files and archive
> terminal state files.
> THis handling will be done consistently for all kinds of actions on datasets.
> As part of this project, we will cleanup un-necessary fields in metada,
> version them and standardize on avro/json.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
