danhuawang opened a new issue, #11781: URL: https://github.com/apache/gravitino/issues/11781
### What would you like to be improved? Group-based authorization (built-in IDP with group→role mapping) introduces significant latency overhead compared to direct user-based authorization for single-table operations. **Performance test results** (30 concurrent users, Auth:ON, Cache:ON, CredentialVending:ON, schema `load3000` with 10,496 tables): | Service | Auth Mode | Samples | RPS | P50 | P75 | P95 | |---------|-----------|---------|-----|-----|-----|-----| | Iceberg REST | User Auth | 4001 | 13.36 | 130ms | 140ms | 180ms | | Iceberg REST | Group Auth | 3673 | 12.27 | 300ms | 360ms | 510ms | | Gravitino | User Auth | 4142 | 13.83 | 65ms | 75ms | 110ms | | Gravitino | Group Auth | 3834 | 12.81 | 210ms | 270ms | 430ms | Key observations: - **Group Auth adds ~130-170ms extra latency per request** compared to User Auth on the same operation. - P50 latency increases by **130% (Iceberg REST)** and **223% (Gravitino)** when switching from User Auth to Group Auth. - Tail latency (P95) is disproportionately affected — P95/P50 ratio grows from 1.38x to 1.70x (Iceberg) and 1.69x to 2.05x (Gravitino), suggesting occasional cache misses or extra lookups in the group→role resolution path. - RPS impact is moderate (~8% drop), but the latency increase is significant for latency-sensitive workloads. The extra latency likely comes from the additional group→role resolution step required on each request when using group-based authorization. ### Test Environment Background - **Metalake:** `irc_test` - **Catalog:** `iceberg` (Iceberg REST service at port 9001, Gravitino service at port 8090) - **Schema:** `load3000` containing **10,496 tables** - **Concurrent users:** 30 (user1 - user30) - **Group Auth setup:** All 30 users belong to a single group `perf_test_group`, which is granted `role_all` (full metalake access). The built-in IDP extension (`org.apache.gravitino.idp.web.rest.feature`) is enabled with `gravitino.authenticators = basic`. - **User Auth setup:** Each user is directly assigned `role_all` plus an individual role (`role1` - `role30`). - **Operation:** Load single table (random table from `load3000` schema) ### How should we improve? Potential solutions: 1. **Cache group→role mapping** — Cache the resolution from group membership to role assignments so it does not require a lookup on every request. 2. **Reduce IDP lookup round-trips** — Batch or pre-resolve user→group→role chain at authentication time rather than at each authorization check. 3. **Session-level authorization context** — Resolve the full permission set once per session/token and reuse it for subsequent requests within the same session. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
