Hi, Le 12/07/2024 à 12:21, Hyunseok Seo a écrit :
*### Why Maintain Empty Directory Markers?* From what I understand, object stores like S3 do not have a concept of directories. The motivation behind maintaining these markers could be to manage the object store as if it were a traditional file system.
Also, to maintain compatibility with other filesystem-like abstractions over S3.
*### Issues with Marker Creation* Users who have raised concerns about the creation of empty directory markers cite the following reasons: - **Increase in Unnecessary Requests [2]**: Creating empty directory markers leads to additional S3 requests, which can increase costs and affect performance.
In which situation does this make a sizable difference in number of requests?
- **File System Consistency Issues [1]**: S3 is designed as an object store, and creating empty directory markers can break the inherent consistency of the file system.
Which "inherent consistency" are we talking about concretely?
*### Proposed Solutions* Issue [1] suggests the following approaches: 1. **Add an Option**: Introduce an option in *S3Options* to control whether empty directory markers are created, giving users the choice.
That sounds ok to me.
3. **Smarter Directory Creation**: Improve the implementation to check for other objects in the same path before creating an empty directory marker.
I don't know what this would achieve, and it would in itself issue additional "unnecessary requests".
Regards Antoine.
