obelix74 opened a new issue, #15314:
URL: https://github.com/apache/iceberg/issues/15314
### Proposed Change
## Summary
This proposal adds a dedicated REST Catalog API endpoint for encryption
key rotation, complementing the `add` and `remove` encryption key actions
introduced in #12987. The
endpoint provides an atomic operation for rotating encryption keys with
"forward-only" semantics.
## Motivation / Use Case
Encryption key rotation is a critical security operation required by
enterprise security policies and compliance frameworks:
### Compliance Requirements
| Framework | Requirement |
|-----------|-------------|
| NIST 800-57 | Cryptoperiod limits based on key usage |
| SOC 2 | Key management controls including rotation |
### Security Use Cases
1. **Scheduled Rotation**: Proactive key rotation per security policy
(e.g., annually, quarterly)
2. **Key Compromise Response**: Immediate rotation when a key is
suspected to be compromised
3. **Personnel Changes**: Rotate keys when key custodians leave the
organization
4. **Cryptoperiod Expiration**: Rotate before keys exceed their
recommended usage lifetime
### Why a Dedicated Endpoint?
While #12987 added `add` and `remove` encryption key actions, key
rotation requires:
- **Atomicity**: Add new key + set as current must be a single atomic
operation
- **Clear Intent**: Rotation is a distinct operation from simply adding a
key
- **Audit Trail**: Security teams need clear rotation events in audit logs
- **Cross-Engine Consistency**: All engines (Spark, Flink, Trino, etc.)
should implement rotation identically
Without a dedicated endpoint, each catalog implementation may implement
rotation differently, leading to divergent behavior across the ecosystem.
There is a RFC underway in the Apache Polaris community to add Table
Encryption and this API is being proposed there. Each catalog might create
their own API. [RFC: Table-Level Encryption Support in Apache
Polaris](https://docs.google.com/document/d/1f4Mgg5W1t4NT6R7KLq5K3S4pHlAwYwXTFwUR9uNNpSU/edit?tab=t.0#heading=h.jhz5bbj54sj1
)
## Proposed API
### Endpoint
POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/encryption/rotate
### Request
{
"new-key-id": "arn:aws:kms:us-east-1:123456789:key/def-456"
}
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `new-key-id` | string | Yes | The new master key ID to use for future
encryption operations |
### Response
{
"previous-key-id": "arn:aws:kms:us-east-1:123456789:key/abc-123",
"current-key-id": "arn:aws:kms:us-east-1:123456789:key/def-456",
"rotated-at": "2025-01-15T10:30:00Z",
"active-key-count": 2
}
| Field | Type | Description |
|-------|------|-------------|
| `previous-key-id` | string | The key ID that was current before
rotation |
| `current-key-id` | string | The new current key ID (from request) |
| `rotated-at` | string (ISO 8601) | Timestamp when rotation was
performed |
| `active-key-count` | integer | Total number of active keys (for reading
historical data) |
### Error Responses
| Status | Error Code | Description |
|--------|------------|-------------|
| 400 | `InvalidKeyId` | The `new-key-id` format is invalid |
| 404 | `TableNotFound` | Table does not exist |
| 409 | `KeyAlreadyCurrent` | The specified key is already the current
key |
| 422 | `TableNotEncrypted` | Table does not have encryption enabled |
| 503 | `KmsUnavailable` | Unable to verify key with KMS |
## Semantics
### Forward-Only Key Rotation
This endpoint implements **forward-only** key rotation.
**Key behaviors:**
1. **New snapshots** use the new key for encryption
2. **Existing snapshots** remain encrypted with their original keys (no
re-encryption)
3. **All historical keys** are retained in the `encryption-keys` array
for reading historical data
4. **Time-travel queries** continue to work (old keys decrypt old
snapshots)
## Alternatives Considered
Without dedicated endpoint:
```
catalog.add_encryption_key(table, new_key_id)
catalog.update_table_properties(table, {"encryption.current-key-id":
new_key_id})
```
**Drawbacks:**
- Not atomic: Failure between calls leaves table in inconsistent state
- Requires client-side coordination
- No clear audit trail for "rotation" as a distinct event
- Different engines may implement differently
---
/cc @rdblue @ggershinsky @amogh-jahagirdar @flyrain
### Proposal document
_No response_
### Specifications
- [ ] Table
- [ ] View
- [ ] REST
- [ ] Puffin
- [ ] Encryption
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
