ashtuchkin opened a new issue, #632: URL: https://github.com/apache/arrow-rs-object-store/issues/632
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** About [a year ago](https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-express-one-zone-append-data-object/), AWS added ability to append data to objects in S3 Express / Directory Buckets: https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-objects-append.html Append is atomic, but has the limit that up to 10000 'parts' can be appended before the file needs to be rewritten. Low-level, appending is implemented by using `x-amz-write-offset-bytes` header, which users need to set to the current size of the file. We've built pretty efficient system using this functionality with official AWS S3 client, but it would be nice if we could do this using object_store somehow. Is it possible? **Describe the solution you'd like** I can see two options: 1. Ideally a top-level "append" operation could be re-introduced (I found it was removed in 0.8). The reasoning for this was that only in-memory and filesystem supported it at the time. Currently there's a bunch of object stores that support it: S3 (directory buckets), Azure, Ceph, Tencent, Alibaba, etc. 2. Use Attributes / Extensions to provide the header directly. Looking at the code I haven't found a good way to do that - would be grateful if you could point me to it! **Describe alternatives you've considered** Using a different client is the only thing that comes to mind. **Additional context** -- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
