Andrey Yarovoy created HDDS-17500:
-------------------------------------
Summary: S3 Gateway should return x-amz-server-side-encryption on
GetObject and HeadObject for encrypted objects
Key: HDDS-17500
URL: https://issues.apache.org/jira/browse/HDDS-17500
Project: Apache Ozone
Issue Type: Bug
Components: s3gateway
Reporter: Andrey Yarovoy
*Problem*
Since HDDS-17498, the S3 Gateway accepts SSE-S3 and SSE-KMS requests on
encrypted (TDE) buckets and returns
`x-amz-server-side-encryption` on the write responses (`PutObject`,
`CopyObject`, `UploadPart`,
`CreateMultipartUpload`). The read responses don't report it: `GetObject` and
`HeadObject` of an object stored
encrypted in such a bucket return no `x-amz-server-side-encryption` header.
AWS S3 returns the header on `GetObject` and `HeadObject` for every encrypted
object: `AES256` for SSE-S3, `aws:kms`
plus `x-amz-server-side-encryption-aws-kms-key-id` for SSE-KMS. Clients and
compliance tools commonly check
encryption with `HeadObject`. For example, `aws s3api head-object` shows
`ServerSideEncryption`, and audit scripts
and data catalogs read it. Against Ozone they see no encryption, although the
object is encrypted at rest, so they
report it as unencrypted.
*Proposed change*
- Return the SSE headers on `GetObject` and `HeadObject` when the object is
encrypted, i.e. it has a
`FileEncryptionInfo`, which objects in encrypted buckets have. Ozone encrypts
with the KMS key, so the natural
values are `x-amz-server-side-encryption: aws:kms` and
`x-amz-server-side-encryption-aws-kms-key-id: <key name>`,
with the key name taken from the object's `FileEncryptionInfo`.
- Objects that are not encrypted keep returning no SSE header.
Points for discussion:
- **Which algorithm to report.** A client that wrote with `AES256` would read
back `aws:kms`. That is accurate for
how Ozone stores the data, but it differs from what the client requested. The
alternative is to store the requested
algorithm in the key's metadata at write time (HDDS-17498 already validates
it), and return that. Objects written
without an SSE header, or before this change, would then fall back to `aws:kms`.
- **HeadObject needs the encryption info.** `GetObject` reads the key with
`getS3KeyDetails`, which returns
`OzoneKeyDetails` with the `FileEncryptionInfo`. `HeadObject` uses
`headS3Object`, which returns an `OzoneKey`
without it. Either `headS3Object` returns the encryption info, or the gateway
uses the bucket's encryption key. The
bucket's key is enough for objects in encrypted buckets, since all of them are
encrypted with it, but it costs a
bucket lookup per `HeadObject`.
- Also report SSE in `GetObjectAttributes`? It has no SSE field in AWS, so
probably not.
- **Multipart objects:** report the same headers for the completed object, and
for `HeadObject` with `partNumber`.
*Testing*
- Unit tests in `hadoop-ozone/s3gateway`: `GetObject` and `HeadObject`, with
and without `partNumber`, return the
SSE headers for an object in an encrypted bucket, and none for an object in a
regular bucket.
- Smoke test (`s3/objectputget.robot`, `encrypted` bucket in `ozonesecure`):
`head-object` returns
`"ServerSideEncryption"` with the expected value, and the KMS key id if
reported.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
