yihua commented on PR #894: URL: https://github.com/apache/incubator-xtable/pull/894#issuecomment-6087743302
A second pass on the boundary between the Hudi commit and the Iceberg commit, in 403b41d: - A failed Iceberg sync now fails the Hudi operation. `TableFormatSync` reports failures in its result rather than throwing, and the hook discarded that result, so a transient Iceberg error left a completed Hudi instant without a snapshot for the next writer to roll back. - Time-based snapshot expiry is off for this target, and the timeline treats a completed instant older than the oldest retained snapshot's instant as completed, so an idle week or an operator-run `expire_snapshots` no longer turns history into pending work. - Rollback is recorded as a forward snapshot from the rollback metadata instead of rewinding to the parent snapshot, which only worked while the rolled-back commit was current. Files of commits Iceberg never recorded are not removed, so the crash path is a plain record, and `IcebergRollbackExecutor` is gone. - Replace commits register files with their size; clustered files reached Iceberg with `file_size_in_bytes` 0 and scanned as empty. `ITIcebergRollbacks` and `ITIcebergTimelineRecovery` cover these, and `testsForClustering` now reads the Iceberg table through Spark. All but the upsert-rollback case fail on the previous commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
