vinishjail97 opened a new issue, #962:
URL: https://github.com/apache/incubator-xtable/issues/962
### Description
A Hudi snapshot sync can fail when the source snapshot has already been
synchronized. The target derives the Hudi instant from the source commit time
and attempts to create that instant again.
This affects an explicit `FULL` sync when the source has not advanced.
Repeating a full sync should safely handle the existing source version rather
than fail while creating an existing timeline file.
### Reproduction
1. Synchronize a source snapshot into a Hudi target.
2. Keep the source commit time and source file list unchanged.
3. Run another snapshot sync against the same target.
4. The target attempts to reuse the previous Hudi instant.
I reproduced the target-level failure with local Hudi tables at versions SIX
and NINE, using Hudi 1.2.1 and Java 11. The test registers an existing file,
then invokes snapshot synchronization with the same source commit time and the
same file list. Both versions fail. Controls that advance the source commit
time pass.
The reproducer invokes the Hudi target directly. A controller-level
regression test for repeated `FULL` synchronization is still needed.
### Actual result
```text
org.apache.hudi.exception.HoodieIOException:
Failed to create immutable file .../<instant>.replacecommit.inflight
Caused by: org.apache.hadoop.fs.FileAlreadyExistsException:
File already exists: .../<instant>.replacecommit.inflight
```
The stack includes:
```text
HudiConversionTarget.completeSync
HudiConversionTarget$CommitState.commit
ActiveTimelineV1.transitionReplaceRequestedToInflight // table version SIX
ActiveTimelineV2.transitionReplaceRequestedToInflight // table version NINE
```
Synchronization fails. I did not observe data loss in these tests.
### Code path
`TableFormatSync.syncSnapshot()` invokes `beginSync()` without filtering an
already synchronized source version. `HudiConversionTarget.beginSync()` derives
the target instant from `table.getLatestCommitTime()`. `CommitState.commit()`
then creates the requested replace commit and transitions it to inflight using
that instant.
### Expected result
Repeated snapshot synchronization of an unchanged source should handle the
existing instant safely. If the target requires metadata repair, the
implementation should preserve that behavior rather than unconditionally skip
every unchanged snapshot.
Regression coverage should include repeated `FULL` synchronization on table
versions SIX and NINE. It should verify the visible files and column statistics
after the repeated operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
