vigneshio opened a new pull request, #4928: URL: https://github.com/apache/polaris/pull/4928
perf: remove redundant exists check before deleteFile (saves cloud API calls) When deleting individual data files asynchronously, the worker first makes a HEAD request (TaskUtils.exists) and then a DELETE request (deleteFile). Cloud storage DELETE operations (like S3 DeleteObject) are generally idempotent and do not fail if the file is absent. Performing a HEAD before every DELETE doubles the number of API calls and latency, severely impacting cloud billing and cleanup performance. Fix: Remove the TaskUtils.exists check and simply call fileIO.deleteFile(file). Added a narrow catch for NotFoundException only to preserve test behavior for InMemoryFileIO (real cloud FileIOs succeed silently on missing objects). ## Checklist - [x] Don't disclose security issues! (contact [email protected]) - [x] Clearly explained why the changes are needed, or linked related issues - [x] Added/updated tests with good coverage, or manually tested (and explained how) - [x] Added comments for complex logic - [x] Updated \`CHANGELOG.md\` (if needed) - [ ] Updated documentation in \`site/content/in-dev/unreleased\` (if needed) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
