[
https://issues.apache.org/jira/browse/HUDI-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519241#comment-17519241
]
sivabalan narayanan commented on HUDI-3799:
-------------------------------------------
We might need to fix how archival handles the empty completed instant.
* Re-create an valid completed instant (could be empty contents too), but lets
not delete the entry. create a empty instant and archive it. so that we have a
record that rollback. completed was part of the timeline.
* Do this for any completed instant and not just for clean and rollback. bcoz,
in hdfs and local, any completed instant could be empty during crash.
> Understand reason behind "Not an avro data file" with hudi
> ----------------------------------------------------------
>
> Key: HUDI-3799
> URL: https://issues.apache.org/jira/browse/HUDI-3799
> Project: Apache Hudi
> Issue Type: Task
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Blocker
> Fix For: 0.11.0
>
>
> We merged [https://github.com/apache/hudi/pull/4016] to tackle "Not an avro
> data file" exception while cleaning or archiving. We need to understand why
> and when such exception happens. and try to mitigate it before happening if
> feasible.
>
> Atleast we should have a good understanding of what are the conditions under
> which this is expected.
>
> Ref: https://github.com/apache/hudi/pull/4016#pullrequestreview-841692564
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
