waterWang opened a new pull request, #17607: URL: https://github.com/apache/iceberg/pull/17607
Fixes #16493 ## Why When using `compareToFileList(Dataset)`, the `file_list_view` is filtered by `startsWith(tableLocation)`. This raw string prefix match incorrectly includes sibling paths that share a prefix with the table location. For example, a table at `s3://bucket/table` would also match `s3://bucket/table-backup/data/file.parquet` or `s3://bucket/table_old/data/file.parquet`, treating files outside the intended directory as in-scope orphan candidates. ## Summary Normalize the location to ensure it ends with a path separator (`/`) before using `startsWith`, and also accept an exact match for the location itself. This ensures that only files actually under the table directory (or equal to the location) are treated as in-scope. ## How to Test New test `testCompareToFileListDoesNotMatchSiblingPaths` verifies that: - Files under `tableLocation + "-backup/"` (sibling prefix) are NOT in scope - Files under `tableLocation + "_old/"` (sibling prefix) are NOT in scope - Files under `tableLocation + "/data/"` are correctly in scope ## Type - [x] Bug fix - [ ] Feature - [ ] Documentation ## Checklist - [x] Changes are tested - [x] Changes follow the project's code style -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
