erickguan opened a new issue, #7825: URL: https://github.com/apache/opendal/issues/7825
### Feature Description Originated from Xuanwo's [comment](https://github.com/apache/opendal/pull/7535#issuecomment-4795815743) in a PR. OpenDAL's `copy` and `rename` has different atomicity depends on the backend support. For example, `fs` backend will attempt `rename` and set metadata. This behavior is similar to `cp --preserve=xattr`. But the two operations in rename are not atomic. (Nor `cp --preserve=xattr` is atmoic.) A common trick for applications is to copy a file to a temporary location and then set xattr. And then move it to a destination. This approach is obviously limited in file system. ### Problem and Solution For OpenDAL, we might need a proper solution to define: 1. Principle: Do we want to guarantee atomicity for `copy` and `rename? 2. API design: How do we define API for atomic operation? ## Principle: Do we want to guarantee atomicity for `copy` and `rename? Some notable implementations: - s3 has strongly consistence for metadata operations along with objects. - Linux doesn't set xattr with files in an atomic operation. We might be able to simulate that but this will result in a performance lost. Some file systems doesn't support copy on write. ## API design 1. Do we want to have `copy` and `rename` to be atomic by default? 2. If impossible, do we add an atomic option? 3. Along with 2, we could have an capability hinting if using atomic property will result in performance loss. ### Additional Context The current behavior depends on backend implementations. ### Are you willing to contribute to the development of this feature? - [ ] Yes, I am willing to contribute to the development of this feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
