tustvold opened a new issue, #5334:
URL: https://github.com/apache/arrow-rs/issues/5334
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
<!--
A clear and concise description of what the problem is. Ex. I'm always
frustrated when [...]
(This section helps Arrow developers understand the context and *why* for
this feature, in addition to the *what*)
-->
The current definition of `ObjectMeta` reflects the information returned by
the various store listing APIs, however, this is an incomplete subset of the
attributes of an object.
A number of requests have been made to allow reading and writing a broader
set of this data:
* Cache-Control - #4498
* Managed encryption keys - #5087
* Content-Type - #3300 #5329
* User Metadata - #4754
**Describe the solution you'd like**
<!--
A clear and concise description of what you want to happen.
-->
I would like a new struct called `Attributes` that is included on
`PutOptions` and `GetResult`, containing an iterable collection of key-value
pairs. The values would be typed as `String`, and the keys would be a new
enumeration called `Attribute`:
```
#[non_exhaustive] // This will allow adding new attributes without it being
a breaking change
enum Attribute {
/// The MIME type of the object
ContentType,
/// An identifier for a managed encryption key
EncryptionKey,
/// User-defined object metadata
Metadata(String),
...
}
```
`ObjectStore` implementations would be expected to error on attributes they
don't understand / aren't supported, and all attributes would be expected to
round-trip. It is an expectation that all first-party stores will support all
`Attribute`.
An optional feature flag would enable support for attributes in
`LocalFileSystem` using `xattr`.
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features
you've considered.
-->
I originally debated adding the ability to specify arbitrary headers in
PutOptions, but this broke encapsulation and made it hard to use `ObjectStore`
without knowing the concrete backing implementation. Whilst the `Attribute`
mechanism doesn't eliminate this issue, as not all implementations will support
all `Attribute`, but it at least avoids implementation code needing to know the
particulars of how Azure vs AWS encode encryption information in
request/response headers, or how Azure uses a different header to write vs read
Content-Disposition (I have no idea why).
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
We can't add `Attributes` to `ObjectMeta` as the listing APIs do not return
this information.
We can't merge `TagSet` with `Attributes` as the tag set is only returned by
special API calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
