dacort opened a new issue, #36275: URL: https://github.com/apache/arrow/issues/36275
### Describe the enhancement requested Today, when using `delete_file` with `S3FileSystem`, it will create an empty file if there's no "directory". Example: Before: ```bash aws s3 ls s3://bucket/prefix/ ``` ``` 2023-06-23 11:43:53 668 prefix/01/file1.json.gz 2023-06-23 11:35:24 1438 prefix/01/file2.json.gz 2023-06-23 10:47:18 819 prefix/01/file3.json.gz ``` _Run `S3FileSystem::delete_file` on file1.json.gz_ After: ``` 2023-06-23 11:51:02 0 prefix/01/ 2023-06-23 11:35:24 1438 prefix/01/file2.json.gz 2023-06-23 10:47:18 819 prefix/01/file3.json.gz ``` This is due to [this line](https://github.com/apache/arrow/blob/ed99693319036f0b8fb6585858541cec7a1268ae/cpp/src/arrow/filesystem/s3fs.cc#L2505) that calls `EnsureParentExists`. Unfortunately, this results in 0-byte files getting created on S3, even if there are other objects in the "directory". It seems like this was somewhat intentional based on [this comment](https://github.com/apache/arrow/pull/5167#discussion_r317699944) in the original PR. That said, I'd love for the ability do disable this. `delete_file` shouldn't create additional files/directories in S3. ### Component(s) Other -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
