jojochuang commented on PR #510: URL: https://github.com/apache/ozone-site/pull/510#issuecomment-5186852936
## Ozone STS vs Amazon S3 STS — divergence and limitations Review of the [Ozone S3 STS guide](https://github.com/apache/ozone-site/blob/master/docs/05-administrator-guide/03-operations/07-s3/02-sts.md) and the [design doc](https://github.com/apache/ozone/blob/master/hadoop-hdds/docs/content/design/ozone-sts.md). ### Overall scope Ozone STS is intentionally **narrow**: it implements a subset of AWS STS centered on **AssumeRole** for temporary S3 credentials. It is **not** a full AWS STS implementation. --- ### Architectural divergences from AWS | Area | AWS | Ozone | |------|-----|-------| | **APIs supported** | Full STS API (`AssumeRole`, `GetSessionToken`, `AssumeRoleWithWebIdentity`, `AssumeRoleWithSAML`, etc.) | **Only `AssumeRole`** | | **Endpoint** | Regional STS endpoint | Dedicated STS port on S3G: `9880` (HTTP) / `9881` (HTTPS), path `/sts` — separate from S3 object API (`9878`) | | **Authorization backend** | AWS IAM | **Apache Ranger only** (Ozone Native Authorizer not supported in v1) | | **Identity model** | IAM users/roles | Kerberos identity + permanent S3 credentials → temporary creds tied to a **Ranger role** | | **Role ARN** | Real AWS account/role | `arn:aws:iam::123456789012:role/<name>` accepted for compatibility; **account ID is ignored**; only the **Ranger role name** matters | | **Session token storage** | AWS-managed | **Stateless, opaque token** encoded server-side (not stored); includes creator Kerberos identity, role, encrypted secret, policy, HMAC | | **Token revocation** | No standard revoke API | **Ozone-specific**: `ozone s3 revokeststoken`; revoking permanent secret revokes all user's STS tokens | | **Prerequisites** | AWS account + IAM | **Kerberos + Ranger + permanent S3 credentials** required | --- ### AssumeRole divergences **Supported optional parameters:** only `DurationSeconds` and `Policy`. **All other AssumeRole fields are rejected** (e.g. `ExternalId`, `PolicyArns`, `Tags`, `TransitiveTagKeys`, `ProvidedContexts`, SAML/WebIdentity params). **Duration:** 15 minutes–12 hours (900–43,200 s); default 3600 s if omitted — matches AWS spec. **Access key prefix:** temporary keys start with `ASIA` (AWS-compatible). --- ### IAM session policy limitations (inline `Policy`) This is the biggest divergence from full AWS IAM: | Feature | AWS IAM | Ozone STS | |---------|---------|-----------| | **`Effect`** | Allow, Deny, etc. | **`Allow` only** — Deny/other effects **rejected** | | **`Resource` ARN** | Many services | Only `arn:aws:s3:::` prefix or `*` — **other prefixes rejected** | | **`Condition` operators** | Many | Only **`StringEquals`** and **`StringLike`** | | **`Condition` keys** | Many | Only **`s3:prefix`** | | **Conditions per statement** | Multiple | **Only one** condition operator per statement | | **`Condition` scope** | Various actions | Only applies to **`s3:ListBucket`** | | **Policy size** | 2048 chars | Same limit | | **Unknown S3 actions** | AWS behavior varies | **Silently ignored** (AssumeRole still succeeds; creds won't grant that action) | | **Unknown/unsupported conditions** | — | **Request rejected** (security: silent ignore would falsely imply restrictions like `sourceIp`) | **Supported action wildcards:** `s3:*`, `s3:Get*`, `s3:Put*`, `s3:List*`, `s3:Create*`, `s3:Delete*`. **Supported S3 actions** (the full set mappable to Ranger) — 15 actions only: - `AbortMultipartUpload`, `CreateBucket`, `DeleteBucket`, `DeleteObject`, `DeleteObjectTagging` - `GetBucketAcl`, `GetObject`, `GetObjectTagging` - `ListAllMyBuckets`, `ListBucket`, `ListBucketMultipartUploads`, `ListMultipartUploadParts` - `PutBucketAcl`, `PutObject`, `PutObjectTagging` Anything outside this set (e.g. `s3:GetAccelerateConfiguration`, `s3:CopyObject`, bucket versioning, lifecycle, etc.) is **unsupported** for session policies. --- ### Ranger-specific divergences (no AWS equivalent) - Permissions are expressed as Ozone **volume/bucket/key** ACL types (`read`, `write`, `create`, etc.) plus **`action-matches`** conditions — not native IAM. - **`ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition`** must be `true` for fine-grained S3 action enforcement; default is `false`. - Session policy permissions = **intersection** of role permissions and inline policy. - **Security warning:** if Ozone STS is enabled but the Ranger action-matcher flag is **not**, tokens can get **broader access than requested** (e.g. `PutObject` token also allowing tagging actions). --- ### Operational limitations - **Disabled by default** (`ozone.s3g.sts.http.enabled=false`). - Requires **secure cluster** (Kerberos + Ranger). - **No Ozone Native Authorizer** support in initial release. - **Multi-tenancy:** volume in policies may differ from `s3v`. - **Ranger policy cache:** up to ~30 s delay after policy changes before AssumeRole reflects them. - **Revocation:** only token creator or S3/tenant admin can revoke; revoked entries cleaned after 12 h. - **Not proxied through Knox** (separate S3G/STS endpoint). --- ### Bottom line Ozone STS is **AWS-compatible where documented**, but only for a **single API (`AssumeRole`)** with a **restricted IAM session policy dialect** and **Ranger-backed roles** instead of IAM. Suitable for data-lake-style temporary, scoped S3 access — **not** a drop-in replacement for AWS STS/IAM. **Unsupported today:** web identity, SAML, `GetSessionToken`, external ID, tag-based session policies, non-S3 resources, Deny statements, IP/VPC conditions, and the vast majority of AWS STS/IAM features. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
