[
https://issues.apache.org/jira/browse/HUDI-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886831#comment-17886831
]
sivabalan narayanan edited comment on HUDI-8282 at 10/4/24 3:48 AM:
--------------------------------------------------------------------
I checked current master code and found only partial support.
[Patch|https://github.com/apache/hudi/pull/9776] were per log file marker was
added.
a. Append handle is emitting per log file marker.
b. Marker based rollback need to be fixed to create rollback request for each
log file. Logic should exaclty be similar to a base file.
c. Marker based reconciliation should also poll for APPEND IoTypes. as of now,
it only checks CREATE and MERGE. and delete spurious files found.
was (Author: shivnarayan):
I checked current master code and found only partial support.
a. Append handle is emitting per log file marker.
b. Marker based rollback need to be fixed to create rollback request for each
log file. Logic should exaclty be similar to a base file.
c. Marker based reconciliation should also poll for APPEND IoTypes. as of now,
it only checks CREATE and MERGE. and delete spurious files found.
> Create per log file marker for 1.x
> -----------------------------------
>
> Key: HUDI-8282
> URL: https://issues.apache.org/jira/browse/HUDI-8282
> Project: Apache Hudi
> Issue Type: Improvement
> Components: writer-core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 1.0.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> This ticket is meant for 1.x. We added per log file marker which was added to
> 0.x branch https://issues.apache.org/jira/browse/HUDI-1517
>
> Commit of interest:
> [https://github.com/apache/hudi/commit/c2c7e0538f8cf3031781ebdd776d1c03bfec3bb3]
>
>
> We might need to add a similar support to 1.x, hence the tracking ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
