GitHub user nevzheng created a discussion: Client-side HTTPS / mTLS authentication as a foundation for RBAC & FGAC
# Research Spike — Client-Side HTTPS / mTLS Authentication as a Foundation for RBAC & FGAC > Status: **Draft for discussion** > Tracking: [#9836](https://github.com/apache/gravitino/issues/9836) · > In-flight PR: [#10975](https://github.com/apache/gravitino/pull/10975) Hi — Jerry asked me to look into this. I put this doc together to capture the current state and prime discussion. Let's align on the direction forward; I'm happy to own the implementation. --- **We need client-side mTLS / HTTPS as a foundation for Role-Based and Fine-Grained Access Control.** Authorization is only as trustworthy as the identity it keys on — so **establishing a strong, non-spoofable identity is the critical prerequisite** for RBAC/FGAC. This spike scopes what that takes in Gravitino: where we stand, what's missing, and the key open decisions. *For how TLS/mTLS actually works, see [Appendix A](#a-how-tls--mtls-works-reference).* --- ## 1. Where we stand today - Authentication is **HTTP-header/token based**. One enforcement point — `AuthenticationFilter` — derives the `Principal` *only* from the `Authorization` header.[^authfilter] Schemes: `NONE, SIMPLE, BASIC, OAUTH, KERBEROS`.[^schemes] - **Server-side HTTPS exists** (keystore, `enableHttps`) — shipped **Dec 2023**.[^https2023] - Jetty can even **require and validate a client cert** at the handshake (`enableClientAuth` + truststore) — but the validated cert identity is **thrown away**; it never reaches the auth layer.[^clientauth] - **`SimpleAuthenticator` trusts the username from a Base64 header with no validation** — the default identity is trivially spoofable. This is the crux for FGAC (§4).[^simple] --- ## 2. Key Use Cases / Product CUJs + Status in Gravitino "Client-side HTTPS auth" spans three escalating layers: | Layer | Meaning | Status | |---|---|---| | **(a)** Server TLS termination | Server presents cert, client trusts it (one-way HTTPS) | ✅ **Shipped** | | **(b)** Client presents cert; server validates at handshake | mTLS as a transport gate | 🟡 **Half done** — server ready; client can't present a cert | | **(c)** Cert identity → Gravitino principal | The cert *is* the login; principal drives authz | ❌ **Missing** — cert never read; SPI is `byte[]`-only | ### Product CUJs | # | Critical User Journey | Needs | Status | |---|---|---|---| | **UC1** | Only trusted machines may connect at all (transport bouncer; token still carries identity) | layer (b) | 🟡 PR [#10975](https://github.com/apache/gravitino/pull/10975) open, conflicting | | **UC2** | The cert *is* the login — no token/password | layer (c) | ❌ Net-new | | **UC3** 🎯 | **RBAC/FGAC keyed on the certificate identity** | layer (c) | ❌ Net-new auth, reuses existing RBAC | | **UC3′** | RBAC keyed on a token; mTLS only hardens transport | layer (b) | ✅ RBAC already works on token principal | **Decision (this spike):** we are building RBAC/FGAC, so we target **UC3 → layer (c)**. Strong, non-spoofable identity is the point (§4). Layer (b)/PR [#10975](https://github.com/apache/gravitino/pull/10975) becomes a prerequisite building block, not the deliverable. --- ## 3. Mermaid Diagrams — per use case ### 3.1 Baseline — what works today (token over one-way HTTPS) ```mermaid sequenceDiagram autonumber participant C as Java / Python Client participant J as Jetty (server TLS) participant F as AuthenticationFilter C->>J: TLS handshake — verify SERVER cert via CA Note over C,J: One-way TLS. Server proves itself. Client stays anonymous at TLS. C->>F: HTTP request + Authorization header (token) F->>F: read header → authenticateToken(byte[]) F-->>C: 200 OK — runs as Principal(user) ``` ### 3.2 UC1 — mTLS transport gate, and where it breaks ```mermaid sequenceDiagram autonumber participant C as Client participant J as Jetty (enableClientAuth=true) participant F as AuthenticationFilter C--xJ: BREAK 1 — client has NO API to present a cert (no SSLContext/keystore) Note over C,J: Server IS ready: a presented cert would be validated against the truststore. J->>F: (suppose validated) peer cert now in TLS session F--xF: BREAK 2 — filter reads ONLY the Authorization header → X509 never read → 401 ``` ### 3.3 UC3 — the full chain the cert must travel to drive RBAC/FGAC ```mermaid sequenceDiagram autonumber participant C as Client (keystore CN=alice) participant J as Jetty (truststore = CA) participant A as Cert Authenticator (NEW = layer c) participant R as RBAC Authorizer (existing subsystem) C->>J: mTLS handshake — present client cert [PR #10975 = BREAK 1] J->>J: validate cert vs CA truststore (exists today) J->>A: hand over the X509 cert [BREAK 2 — never happens today] A->>A: map cert subject CN=alice → Principal(alice) A->>R: authenticated Principal(alice) R->>R: look up alice's roles + privileges (RBAC policy) R-->>C: allow / deny per policy Note over A,R: Same RBAC engine as today. Layer (c) only changes WHERE the principal comes from: the cert instead of a token. ``` --- ## 4. Why We Need This — establishing identity for RBAC + FGAC **FGAC/RBAC is only as trustworthy as the identity it keys on.** Today Gravitino's `SimpleAuthenticator` takes the username from a Base64 header and trusts it with **no validation** — anyone can claim to be `alice`. Building row/column masking and fine-grained privileges on top of a spoofable principal is security theater: a perfect policy engine is still bypassed by setting a header. mTLS cert-as-identity delivers a **non-repudiable principal proven by possession of a private key** — that is why client-cert auth is a foundation, not a feature. ### Open question — who does FGAC govern? | Principal type | Best identity foundation | |---|---| | **Services / data-plane clients** (Spark, Trino, jobs) | **mTLS certs** — ideal; machine identity, no IdP round-trip | | **Human analysts** (per-person masking) | **OIDC/OAuth** usually better — cert-per-human is painful to issue/rotate | → If FGAC is primarily **service/machine identity**, certs are *the* foundation (layer c). If **human end-users**, certs are *a* hardening layer alongside OIDC. **(Open question — see §D.)** ### Recommended design seam Follow the **Kafka model**: **the cert supplies identity only; roles/privileges stay in Gravitino's RBAC store.** Do *not* encode roles in cert OUs — that couples authz to PKI and forces cert re-issuance on every role change. Cert = *who*; Gravitino RBAC = *what*. --- ## 5. Prior Art & Related Work ### 5.1 Gravitino issues & PRs All related work found in `apache/gravitino`, grouped by how directly it bears on client-cert / mTLS identity. Layer refs are (a) server TLS, (b) client presents cert, (c) cert-as-identity. **Core thread — client-side certificate / mTLS** | Ref | Type · State | What it does | Bearing | |---|---|---|---| | [#9836](https://github.com/apache/gravitino/issues/9836) | Issue · **open** | Tracking issue: add a `tlsConfigurer` so trusted clients are issued/present SSL certs and *"only authorized clients can access the service."* Cites [iceberg#13190](https://github.com/apache/iceberg/pull/13190). | **Primary.** Scope-as-written = layer **(b)** transport gate; does not call out cert-as-identity or RBAC mapping | | [#10975](https://github.com/apache/gravitino/pull/10975) | PR · **open, CONFLICTING** | Implements #9836: a `TLSConfigurer` + `.withTlsConfigurer()` on the client, wired into the HC5 connection manager; config template + `TestHTTPClient`. | **Primary.** Delivers layer **(b) / BREAK 1** (Java only). Does *not* touch server cert-reading, the SPI, or mapping (BREAK 2). Detail below ⬇ | | [#936](https://github.com/apache/gravitino/issues/936) | Issue · **open** (dormant, 2023) | Oldest request: *"GravitinoClient doesn't support to pass the certificate … it's much safer."* No design or PR. | **Primary (historical).** Original statement of need; superseded in practice by #9836/#10975, never closed | **Foundation — server-side TLS (shipped)** | Ref | Type · State | What it does | Bearing | |---|---|---|---| | [#41](https://github.com/apache/gravitino/issues/41) / [#860](https://github.com/apache/gravitino/pull/860) | Issue closed / PR **merged** (Dec 2023) | Server-side HTTPS for Jetty: `enableHttps` + keystore, plus `enableClientAuth` + truststore (`setNeedClientAuth`). | **Foundation.** Server halves of layers **(a)** and **(b)** already exist — including handshake-level client-cert *validation* | **Adjacent — authentication / identity design (context for layer c)** | Ref | Type · State | What it does | Bearing | |---|---|---|---| | [#10881](https://github.com/apache/gravitino/issues/10881) / [#10883](https://github.com/apache/gravitino/pull/10883) | Issue closed / PR **merged** | *"Local authentication in Gravitino"* — design doc + feature for local-account authentication (a new authenticator scheme). | **Precedent** for adding an authenticator; the same `Authenticator`/`AuthenticatorType` surface a cert authenticator (Gap 3) would extend | | [#11622](https://github.com/apache/gravitino/pull/11622) | PR · **merged** | MCP-server auth/authz/audit foundation: forwards the `Authorization` header **verbatim**, supporting OAuth2 Bearer + simple `Basic`; adds per-request authorization + audit. | **Reinforces the gap** — Gravitino's newest identity path is still header/token, per-request; no cert path | | [#11663](https://github.com/apache/gravitino/pull/11663) | PR · **merged** | Docs: MCP-server authentication, **per-request identity, TLS**, and audit logging. | **Context** — documents the current auth + TLS story to build on | **Peripheral — security docs & downstream transport** | Ref | Type · State | What it does | Bearing | |---|---|---|---| | [#10732](https://github.com/apache/gravitino/pull/10732) | PR · **open** | Adds design documents for **sensitive-data management** (security). | **Context** — where data-protection/FGAC design is landing | | [#848](https://github.com/apache/gravitino/issues/848) | Issue · closed | Added the **Security documentation page** (HTTPS / auth how-tos). | **Context** — the existing server-HTTPS docs | | [#7575](https://github.com/apache/gravitino/issues/7575) | Issue · **open** | Q&A: connecting a **Kafka catalog** over SSL/SASL_SSL (downstream, via `gravitino.bypass.`). | **Out of scope** — downstream catalog transport, not client→Gravitino auth | **[#10975](https://github.com/apache/gravitino/pull/10975) in detail** (the active implementation): contributors first concluded mTLS was *already* supported server-side (Jetty validates client certs) and initially submitted **only tests** proving that; the work was then redirected to the **client** side — *"You need to modify the client code"* — citing [iceberg#13190](https://github.com/apache/iceberg/pull/13190). They added the `TLSConfigurer` + `.withTlsConfigurer()` builder, `gravitino.conf.template` entries, and `TestHTTPClient` cases. **Delivers layer (b) / BREAK 1 (Java only); does not touch BREAK 2 / layer (c).** Currently needs conflict resolution + review. ### 5.2 External reference implementations (cert → principal → authz) Not Gravitino work — mature designs to borrow from for layer (c). | Engine | Mechanism | What it does | Relevance | |---|---|---|---| | **Apache Kafka** | `ssl.principal.mapping.rules` | Regex-maps cert **DN → principal**; ACLs key on it; roles live in the broker | Canonical cert-as-identity-for-authz; the model to mirror | | **Kubernetes** | cert **`CN` = user, `O` = group(s)** | API server derives username + groups from the cert, then applies RBAC | The *alternative* — carry groups in the cert | | **Trino** | `authentication.type=CERTIFICATE` | Client-cert auth + cert→user mapping in an HTTP query engine | Closest architectural analog | | **PostgreSQL** | `cert` auth + `pg_hba`/ident maps | Maps cert CN → db user with configurable rules | Mature DN→user mapping precedent | | **[Iceberg REST #13190](https://github.com/apache/iceberg/pull/13190)** | client `SSLContext` / TLS configurer | Lets the Iceberg REST *client* present certs / use custom trust | The layer-**b** reference cited by [#9836](https://github.com/apache/gravitino/issues/9836); template for [#10975](https://github.com/apache/gravitino/pull/10975) | --- ## 6. Gap Analysis (code-grounded) ### 6.1 What already exists (do not rebuild) - **Server-side TLS termination** — `enableHttps` + keystore. `JettyServerConfig.java` (`123–167`), `JettyServer.java` (`391–425`). - **Server-side handshake client-cert validation** — `enableClientAuth` → `setNeedClientAuth(true)` + truststore. `JettyServerConfig.java:183`, `190–207`; `JettyServer.java:427–430`. - **A full RBAC subsystem keyed on the authenticated `Principal`** — users/groups, roles, privileges on securable objects, a REST permission API, request-time authorization filters, and a pluggable authorizer (incl. an Apache Ranger plugin). It consumes whatever principal auth produces.[^rbac] ### 6.2 What is missing (the real gaps) **Gap 1 — Client cannot *source* a cert to present in the handshake (TLS layer / BREAK 1).** A client-side config gap, not a missing server endpoint: no API to point the client at a keystore, build an `SSLContext` with a `KeyManager`, and hand it to the TLS stack. `HTTPClient.java` builds the HC5 client with **no `SSLContext`** (`~123`, conn-mgr `748–761`); `GravitinoClientConfiguration` has **no TLS keys**. Python `http_client.py:182` uses `build_opener()` with **no `ssl.SSLContext`**. → *Partially addressed for Java by [#10975](https://github.com/apache/gravitino/pull/10975); Python uncovered.* **Gap 2 — Server discards the validated cert (app layer / BREAK 2).** `AuthenticationFilter` reads **only** the `Authorization` header (`83–101`) and never touches the request's `X509Certificate` attribute. Repo-wide: **zero `X509Certificate` references** in `server/`, `server-common/`, `core/`, `common/`, `clients/`. → even with `enableClientAuth=true`, a request with no `Authorization` header is rejected (`401`). **Not addressed by [#10975](https://github.com/apache/gravitino/pull/10975).** **Gap 3 — The `Authenticator` SPI cannot express cert auth.** `Authenticator` is `authenticateToken(byte[]) → Principal` only — no access to the request or TLS peer certs. `AuthenticatorType` (`common/.../auth/AuthenticatorType.java:23`) has no `CERTIFICATE`; `AuthenticatorFactory:40` has no mapping. → a request/peer-cert-aware auth path is required (core architectural change). **Not addressed by [#10975](https://github.com/apache/gravitino/pull/10975).** **Gap 4 — No cert → principal mapping.** Nothing maps a cert subject (`DN`/`CN`/`SAN`) to a Gravitino user. Required for cert-as-identity; needs configurable rules (Kafka-style). **Not addressed by [#10975](https://github.com/apache/gravitino/pull/10975).** **Gap 5 — Weak default identity undermines FGAC.** `SimpleAuthenticator` trusts the header username with no validation — why FGAC needs a strong, verifiable principal (§4). ### 6.3 Accurate one-line status > Server-side TLS **and** handshake-level client-cert *validation* already > exist. [#10975](https://github.com/apache/gravitino/pull/10975) adds the Java > client's ability to *present* a cert (layer b). **Everything that turns a > validated cert into an authz-usable principal — reading the cert (Gap 2), an > SPI that can express it (Gap 3), and DN→principal mapping (Gap 4) — does not > exist.** That trio is the actual work for an RBAC/FGAC foundation. --- # Appendix ## A. How TLS / mTLS works (reference) **One-way vs mutual.** Normal HTTPS = only the **server** presents a cert; the client verifies it against a CA and stays anonymous at the TLS layer (what Gravitino has today). **mTLS** = the **client also** presents a cert, validated by the server — both ends identified before any HTTP flows. **Why the whole connection stays trusted.** The cert is presented once, in the TLS handshake, which does two things at once: (1) **authenticates the peer** — the client signs part of the handshake with its private key; the server checks the cert is signed by a CA in its truststore *and* that the peer holds the private key (validated locally — no CA call per request); (2) **derives session keys** via a key exchange signed with that private key, so the keys are bound to the authenticated identity. Every later record is AEAD-encrypted with those keys, so any record that verifies provably came from the handshake peer. → **identity is sticky to the connection: no token, nothing re-signed per request.** ``` cert (+ private-key proof) → handshake → session keys (bound to identity) → every record authenticated mTLS (cert = identity): handshake(cert) ─┬─ req1 ── req2 ── req3 (identity sticky to connection) bearer token: handshake ────────┴─ req1+tok ── req2+tok (identity re-sent every request) ``` **It's all the TLS layer — HTTP never sees the cert.** Only two places leave pure TLS, neither HTTP-native: the server *consuming* the validated cert is an **app-layer** step (Gap 2); and the `X-SSL-Client-Subject` header only appears in the **proxy case**, when TLS terminates upstream at a load balancer. (One connection = one identity — a proxy multiplexing many users over one TLS connection needs app-layer identity per request.) **PKI trust material — who holds what:** ```mermaid flowchart TB CA["CERTIFICATE AUTHORITY (CA)<br/>trust anchor — signs every cert below"]:::ca CA -->|"signs & issues"| CC["CLIENT KEYSTORE<br/>client cert + private key<br/>subject: CN=alice"]:::client CA -->|"signs & issues"| SC["SERVER KEYSTORE<br/>server cert + private key"]:::server CA -->|"CA cert = trust anchor"| ST["SERVER TRUSTSTORE<br/>validates CLIENT certs (enableClientAuth)"]:::server CA -->|"CA cert = trust anchor"| CT["CLIENT TRUSTSTORE<br/>validates the SERVER cert"]:::client classDef ca fill:#0d3b66,color:#fff,stroke:#08243f classDef client fill:#264653,color:#fff,stroke:#16303a classDef server fill:#3d405b,color:#fff,stroke:#26283b ``` **keystore** = "certs I present to prove myself." **truststore** = "the CA I trust to vouch for the other side." ## B. Glossary - **CA:** trust anchor that signs certs; both sides trust it. · **Keystore:** holds *my* cert + private key. · **Truststore:** holds the CA cert(s) I trust for the *other* side. · **DN / CN / SAN:** cert identity fields (source for the mapped principal). · **mTLS:** both ends present certs. · **FGAC:** fine-grained (row/column/object) access control. ## C. Scope of work if layer (c) is chosen 1. **Client keystore support** — present the cert ([#10975](https://github.com/apache/gravitino/pull/10975)'s `TLSConfigurer`; prerequisite; extend to Python). 2. **Server reads the validated cert** off the TLS session in `AuthenticationFilter` (Gap 2). 3. **SPI change** — request/peer-cert-aware authenticator path (Gap 3; touches all authenticators). 4. **Cert → principal mapping** — `CN`/SAN → user, configurable rules (Gap 4). 5. **Seam to RBAC/FGAC** — mapped principal flows into the existing authorizer; roles stay server-side. ## D. Open decisions to resolve 1. **Service vs human identity** — machines (certs = foundation) or humans (OIDC primary, certs a hardening layer)? 2. **Layer (b) vs (c)** — (c) chosen; is [#10975](https://github.com/apache/gravitino/pull/10975) the accepted prerequisite, or a fresh design? 3. **Roles in cert vs store** — Kafka-style (identity-only cert, roles in store — recommended) vs Kubernetes-style (`O=group` in cert)? 4. **TLS termination topology** — terminate at Gravitino, or behind a proxy (header-forwarded identity)? --- [^authfilter]: `AuthenticationFilter.doFilter` reads the credential solely from the `Authorization` header ([`server-common/.../authentication/AuthenticationFilter.java:83`](https://github.com/apache/gravitino/blob/main/server-common/src/main/java/org/apache/gravitino/server/authentication/AuthenticationFilter.java#L83)), converts it to `byte[]`, and derives the `Principal` only via `authenticator.authenticateToken(authData)` (`:91–95`). A null principal is rejected as `UnauthorizedException` (`:100–101`). No other request state (TLS session, client cert) is consulted. [^schemes]: The scheme set is a closed enum — exactly `NONE, SIMPLE, BASIC, OAUTH, KERBEROS` — at [`common/.../auth/AuthenticatorType.java:25–37`](https://github.com/apache/gravitino/blob/main/common/src/main/java/org/apache/gravitino/auth/AuthenticatorType.java#L25-L37). [^https2023]: Commit `68af8d674`, **2023-12-04** — *"[#41] feat(server): Add https support for Jetty server (#860)"* ([#41](https://github.com/apache/gravitino/issues/41) / [PR #860](https://github.com/apache/gravitino/pull/860)). `enableHttps` at `JettyServerConfig.java:123` plus keystore keys. [^clientauth]: `enableClientAuth` (`JettyServerConfig.java:183`) + truststore keys (`:190–207`), loaded only when `enableHttps && enableClientAuth` (`:400–402`); `JettyServer.java:427` calls `sslContextFactory.setNeedClientAuth(true)` with `setTrustStore*` (`:428–430`) — so Jetty validates the client cert at the handshake. **But** a repo-wide search finds **zero `X509Certificate` references** in non-test `server/ server-common/ core/ common/ clients/`, and `AuthenticationFilter` never reads the cert (see [^authfilter]) — the validated identity is discarded. (Introduced in the same 2023 commit as HTTPS.) [^simple]: `SimpleAuthenticator`'s own class javadoc: *"SimpleAuthenticator will use the identifier provided by the user without any validation"* ([`server-common/.../authentication/SimpleAuthenticator.java:32–34`](https://github.com/apache/gravitino/blob/main/server-common/src/main/java/org/apache/gravitino/server/authentication/SimpleAuthenticator.java#L32-L34)); it Base64-decodes the Basic header and returns `new UserPrincipal(userInformation[0], authData)` (`:66`) with no credential check. [^rbac]: Gravitino already ships an RBAC subsystem: a model of users, groups, roles, and privileges on securable objects (`api/.../authorization/` — `Role`, `Privilege`/`Privileges` incl. `USE_CATALOG`, `CREATE_TABLE`, `MODIFY_TABLE`, `SELECT_TABLE`; `SecurableObject`, `User`, `Group`); core managers (`core/.../authorization/AccessControlManager.java`, `PermissionManager.java`, `UserGroupManager.java`); a REST permission API (`server/.../web/rest/PermissionOperations.java`); request-time authorization filters (`server/.../web/filter/authorization/*Executor.java`); and a pluggable authorizer with an Apache Ranger plugin (`authorizations/authorization-ranger/`). Authorization keys on the authenticated principal via `PrincipalUtils.getCurrentPrincipal()` — the exact input cert-as-identity would supply. GitHub link: https://github.com/apache/gravitino/discussions/11998 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
