Revanth14 opened a new pull request, #2187: URL: https://github.com/apache/iceberg-go/pull/2187
## Problem On flat object stores (S3, GCS, in-memory), `blobfs.FileIO.WalkDir` reports a subdirectory's trailing-slash marker as a file named after that subdirectory. Walking `tbl` with a `tbl/data/` marker yields `tbl/data/data`, a path that doesn't exist. Orphan cleanup would report it as an orphan and try to delete it. ## Fix `walkFS.ReadDir` now skips the listed object whose key equals the prefix of the directory being read. Under a `/` delimiter, that object can only be the directory's own `name/` marker. A directory that holds only its marker reads as empty rather than missing, so walking an empty marker directory doesn't fail. This check also covers the walk root's marker, so `isDirectoryMarker` and its skip in the `WalkDir` callback are removed. A non-empty object stored at a `dir/` key is no longer reported. Previously it appeared under a path that doesn't exist, which cleanup couldn't act on anyway. ## Testing Added `TestBlobFileIOWalkDirSkipsSubdirectoryMarkers`, covering a populated subdirectory marker, an empty one (`tbl/empty/`), and a directory with no marker. Removing either the marker skip or the empty-directory guard makes it fail, along with the existing `TestBlobFileIOWalkDirSkipsDirectoryMarker`. Closes #2185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
