amaliujia commented on code in PR #10742: URL: https://github.com/apache/ozone/pull/10742#discussion_r3576712351
########## hadoop-hdds/docs/content/design/common-auth-service.md: ########## @@ -0,0 +1,656 @@ +--- +title: Ozone Auth Service — Protocol-Neutral Pluggable Authentication and Authorization +summary: A standalone service that unifies all identity validation and policy evaluation for Ozone +date: 2026-07-13 +jira: HDDS-15845 +status: proposed +author: Abhishek Pal +--- +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# Ozone Auth Service — Design Doc + +## Table of Contents +1. [Motivation](#1-motivation) +2. [Proposal](#2-proposal) + * [2.1 Token Format](#21-token-format) + * [2.2 OMRequest Wire Extension](#22-omrequest-wire-extension) + * [2.3 Authentication Provider Plugin Interface](#23-authentication-provider-plugin-interface) + * [2.4 Authorization Backend Interface](#24-authorization-backend-interface) + * [2.5 Token Signing and Verification](#25-token-signing-and-verification) + * [2.6 OM Integration](#26-om-integration) + * [2.7 RPC Protocol Support](#27-rpc-protocol-support) + * [2.8 S3 Gateway Migration](#28-s3-gateway-migration) + * [2.9 REST/gRPC Gateway for Lightweight Clients](#29-restgrpc-gateway-for-lightweight-clients) + * [2.10 Edge Cases and Security Considerations](#210-edge-cases-and-security-considerations) +3. [Upgrades](#3-upgrades) +4. [Industry Patterns](#4-industry-patterns-request-bound-tokens-in-distributed-systems) +5. [Future Work](#5-future-work) + +--- + +## 1. Motivation + +Ozone currently fragments authentication and authorization across multiple client/protocol paths with no unified control plane: + +| Access Path | Authentication | Authorization | +|:------------|:---------------|:--------------| +| Hadoop RPC (CLI, OzoneFS) | Kerberos SASL at transport layer (`UserGroupInformation`) | `IAccessAuthorizer.checkAccess()` in OM | +| S3 Gateway | AWS SigV4 parsed in `AWSSignatureProcessor`, re-verified in OM | Same `IAccessAuthorizer` via `RequestContext` | +| Delegation Tokens | `OzoneDelegationTokenSecretManager` inside OM | Token carries identity; same authorizer | +| STS/AssumeRole | `AssumeRoleRequest` processed in OM | Session policy intersected with ACLs | + +### Problems with the current architecture + +#### a) Authentication is locked to transport +Kerberos is negotiated at the Hadoop RPC connection level. gRPC does not support Kerberos/SASL natively, so gRPC-based services require a different authentication mechanism. Clients without a Kerberos ticket or S3 keys have no path to authenticate. + +#### b) No extensible provider model +Supporting OAuth/OIDC or any non-Kerberos credential requires modifying `OMClientRequest.getUserInfo()` and the OM request handling pipeline. There is no plugin interface for authentication providers. + +#### c) S3 signature verification is duplicated +S3 Gateway parses the signature (`AWSSignatureProcessor`), then OM re-verifies it against the stored secret via `S3SecretManager`. The signature is computed twice for every S3 request. + +#### d) OM performs identity validation in the Ratis state machine path +This constrains OM's ability to call external identity stores or policy engines during request processing. External calls from the Ratis apply path would break determinism. + +#### e) Policy enforcement is fragmented +S3 Gateway, OM, and delegation token managers each log authentication events separately. There is no unified audit stream for all authentication decisions across all access paths. + +--- + +## 2. Proposal + +Introduce a standalone **Ozone Auth Service** — a separate process that becomes the single owner of client-facing authentication and authorization for all Ozone metadata operations. + +``` +┌────────────┐ ┌───────────────┐ ┌────────────────┐ +│ Client │────▶│ Auth Service │────▶│ Ozone Manager │ +│ │ │ │ │ │ +│ Presents │ │ 1. Validate │ │ 1. Verify │ +│ credential │ │ credential │ │ token sig │ +│ + target │ │ 2. Evaluate │ │ 2. Check │ +│ operation │ │ policy │ │ binding │ +│ │ │ 3. Issue │ │ 3. Execute │ +│ │ │ token │ │ operation │ +└────────────┘ └───────────────┘ └────────────────┘ +``` + +The Auth Service validates any supported credential type, evaluates authorization policy (Ranger or native ACLs), and issues a short-lived cryptographically-signed `OzoneAuthToken` bound to the specific operation and resource. OM verifies the token locally — no network calls, preserving Ratis determinism. + +### Module Structure + +- **`hadoop-ozone/ozone-auth-common`**: Shared library containing the `OzoneAuthToken` protobuf, `TokenVerifier`, and provider plugin interfaces. Depended on by both Auth Service and OM. +- **`hadoop-ozone/ozone-auth-service`**: Standalone service process with gRPC/REST server, provider implementations, authorization backends, and audit logging. + +--- + +### 2.1 Token Format + +The `OzoneAuthToken` is a protobuf message that binds an authenticated identity to a specific operation and resource. + +```protobuf +syntax = "proto2"; +option java_package = "org.apache.hadoop.ozone.protocol.proto"; +option java_outer_classname = "OzoneAuthTokenProtos"; +package hadoop.ozone; + +message OzoneAuthTokenProto { + // The authenticated principal (e.g., "[email protected]") + optional string subject = 1; + + // Which auth provider authenticated this principal + // Values: "kerberos", "s3", "oidc", "sts" + optional string authProvider = 2; + + // The Auth Service instance that issued this token + optional string issuer = 3; + + // The target OM cluster this token is valid for + optional string audience = 4; + + // The operation type this token authorizes (matches OMRequest.Type enum name) + optional string cmdType = 5; + + // The resource path this token covers + // Supports trailing /* for prefix scopes (e.g., "/vol1/bucket1/*") + optional string resourceScope = 6; + + // The ACL action permitted: "READ", "WRITE", "CREATE", "DELETE", "LIST" + optional string allowedAction = 7; + + // Expiration time in milliseconds since epoch + optional uint64 expiryMs = 8; + + // Unique token ID (UUID) for replay protection + optional string tokenId = 9; + + // Cryptographic signature over all fields above + optional bytes signature = 10; + + // SHA-256 hash of the canonical request parameters + // Prevents parameter tampering after token issuance + optional string canonicalRequestHash = 11; + + // Session policy for STS-style scoped tokens (JSON) + optional string sessionPolicy = 12; + + // Identity of the calling service (e.g., "mcp-server-prod") + optional string sourceService = 13; + + // Policy version at the time of authorization + // OM can reject tokens with epoch older than a configured revocation point + optional uint64 policyEpoch = 14; + + // Resolved group memberships for the subject + repeated string groups = 15; + + // Signature algorithm used (default: "HMAC-SHA256") + optional string signatureAlgorithm = 16; + + // Key ID used for signing (enables key rotation) + optional string signingKeyId = 17; +} +``` + +``` +Note: All fields are optional because proto3 does not support the required keyword. +Field presence is enforced at the application level during token construction and +verification. +``` + +#### Token field semantics + +| Field | Verification at OM | Purpose | +|:------|:-------------------|:--------| +| `subject` | Used as request identity | Maps to `UserInfo.userName` | +| `audience` | Must match this OM's cluster ID | Prevents cross-cluster token reuse | +| `cmdType` | Must match `OMRequest.cmdType` | Prevents operation substitution | +| `resourceScope` | Must cover the requested resource path | Prevents resource escalation | +| `expiryMs` | Must be in the future (with clock skew tolerance) | Natural revocation | +| `tokenId` | Must not be replayed (write ops) | Prevents duplicate mutations | +| `signature` | HMAC verified with shared key | Integrity and authenticity | +| `policyEpoch` | Must be ≥ configured floor | Invalidates tokens from stale policies | + +#### Token size estimate + +A typical token with 3 groups and no session policy is ~350-500 bytes serialized. This is added to every `OMRequest` as an optional field. + +--- + +### 2.2 OMRequest Wire Extension + +The existing `OMRequest` protobuf gains one new optional field: + +```protobuf +// In OmClientProtocol.proto +import "OzoneAuthToken.proto"; + +message OMRequest { + // ... existing fields (1-95) unchanged ... + + // Auth token from the Ozone Auth Service. + // When present, OM uses this for identity and skips re-authentication. + // When absent, OM falls back to existing Kerberos/S3 auth mechanisms. + optional hadoop.ozone.OzoneAuthTokenProto authToken = 120; +} +``` + +**Backward compatibility:** The field is optional. Existing clients that do not set it continue to work via Kerberos UGI or `S3Authentication`. OM checks for token presence before falling back. + +--- + +### 2.3 Authentication Provider Plugin Interface + +Each provider handles one credential type and is loaded by the Auth Service at startup. + +```java +/** + * Plugin interface for authentication providers. + * Implementations MUST be thread-safe after init() returns. + * + * Lifecycle: construct → init(conf) → authenticate() [concurrent] → close() + */ +public interface AuthenticationProvider extends Closeable { + + /** Human-readable name (e.g., "kerberos", "s3", "oidc"). */ + String name(); + + /** Which credential types this provider handles. */ + Set<CredentialType> supportedTypes(); + + /** One-time initialization (load keytabs, configure JWKS, etc.). */ + void init(OzoneConfiguration conf) throws AuthenticationException; + + /** Authenticate a credential. Returns identity on success. */ + AuthResult authenticate(CredentialEnvelope envelope) Review Comment: Actually found it is documented below -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
