Andrey Yarovoy created HDDS-17498:
-------------------------------------
Summary: S3 Gateway should reject requests for server-side
encryption instead of silently ignoring them
Key: HDDS-17498
URL: https://issues.apache.org/jira/browse/HDDS-17498
Project: Apache Ozone
Issue Type: Bug
Components: s3gateway
Reporter: Andrey Yarovoy
*Problem*
The S3 Gateway doesn't support S3 server-side encryption (SSE): `S3.md` lists
it as not supported. It doesn't reject
requests for it either. No code in `hadoop-ozone/s3gateway` reads the
`x-amz-server-side-encryption*` headers, and
`ObjectAttributesHandler` notes that "SSE-C encryption headers are also not
supported and are silently ignored".
So a client that asks for its data to be encrypted at rest gets a success
response, and the object is stored as
written, which is unencrypted unless the bucket is an encrypted (TDE) bucket.
For example:
- `PutObject` with `x-amz-server-side-encryption: AES256` or `aws:kms` returns
200 and stores the object in plaintext;
- `PutObject` with SSE-C
(`x-amz-server-side-encryption-customer-algorithm/-key/-key-MD5`) returns 200
and the
customer key is discarded. A later `GetObject` without the key also succeeds,
while S3 requires the key to read an
SSE-C object.
The client has no way to tell that its request wasn't honoured. Applications
and compliance checks that rely on
requesting SSE therefore assume the data is encrypted at rest when it isn't.
AWS S3 fails such requests when it can't
honour them, for example an unknown algorithm or a missing or invalid customer
key, rather than storing the data
unencrypted.
The bucket-level API is already handled: `PutBucketEncryption` and
`GetBucketEncryption` (`?encryption`) return
`NotImplemented` through `BucketOperationNotImplementedHandler`. Only the
per-request headers are ignored.
*Proposed change*
Reject object requests that ask for server-side encryption with the existing S3
`NotImplemented` error (501), the
same way the gateway already rejects other unsupported features.
Headers to check:
- `x-amz-server-side-encryption` (SSE-S3 / SSE-KMS);
- `x-amz-server-side-encryption-aws-kms-key-id`,
`x-amz-server-side-encryption-context`,
`x-amz-server-side-encryption-bucket-key-enabled`;
- `x-amz-server-side-encryption-customer-algorithm`, `-customer-key`,
`-customer-key-MD5` (SSE-C);
- `x-amz-copy-source-server-side-encryption-customer-*` (SSE-C copy source).
Operations to check: `PutObject`, `CopyObject`, `CreateMultipartUpload`,
`UploadPart` and `UploadPartCopy`. For SSE-C,
also the read operations that carry the customer key: `GetObject`, `HeadObject`
and `GetObjectAttributes`.
- **Compatibility.** Clients that send these headers today, and work because
the headers are ignored, will start to
fail. An example is S3A with `fs.s3a.encryption.algorithm` set. That is the
intent: the failure is visible, rather
than the data silently not being encrypted. If a transition is needed, a
gateway setting could keep the old
behaviour, but it should default to rejecting.
- Update `S3.md`, and the `ObjectAttributesHandler` comment, to say that SSE
requests are rejected.
*Testing*
- Unit tests in `hadoop-ozone/s3gateway`: each listed header on each listed
operation returns `NotImplemented`, and
nothing is written (no key, no multipart upload created). The same requests
without the headers still succeed.
- If SSE-S3/KMS is accepted for encrypted buckets: such a request succeeds on
an encrypted bucket and is rejected on a
regular one.
- Smoke test (`hadoop-ozone/dist/src/main/smoketest/s3`): `aws s3api put-object
--server-side-encryption AES256` and
`--sse-customer-algorithm AES256 --sse-customer-key ...` fail with
`NotImplemented`.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
