zeroshade commented on code in PR #1326:
URL: https://github.com/apache/iceberg-go/pull/1326#discussion_r3522376215
##########
catalog/hadoop/hadoop.go:
##########
@@ -568,10 +612,32 @@ func (c *Catalog) CommitTable(ctx context.Context, ident
table.Identifier, reqs
return updated, newMetaPath, nil
}
-func (c *Catalog) commitMetadataFile(ident table.Identifier, tempPath,
metaPath string, conflictErr error) error {
- if err := c.filesystem.RenameNoReplace(tempPath, metaPath); err != nil {
+func (c *Catalog) commitMetadataFile(ident table.Identifier, version int,
tempPath, metaPath string, conflictErr error) error {
+ claimPath := c.metadataVersionClaimPath(ident, version)
+ if err := c.filesystem.RenameNoReplace(tempPath, claimPath); err != nil
{
Review Comment:
[MAJOR] A stale `.claim` file can permanently block this metadata version.
The claim is created here with `RenameNoReplace(tempPath, claimPath)`, then is
only removed by the deferred cleanup when `removeClaim` remains true or
intentionally retained after the final rename. If the process dies after taking
the claim but before publishing the final metadata file,
`vN.metadata.json.claim` remains with no final `vN.*.metadata.json`, so every
future attempt for that version hits `fs.ErrExist` and returns a conflict even
though `metadataVersionLocation` finds no actual metadata file. Please make
stale claims recoverable: when `claimPath` already exists, check whether any
final metadata variant exists via `metadataVersionLocation`; if none exists,
treat the claim as abandoned by reclaiming/removing it, or encode enough
information to distinguish in-progress vs abandoned. Add a regression test that
leaves a `vN.metadata.json.claim` with no `vN.*.metadata.json` and verifies
recovery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
