lasdf1234 opened a new issue, #11791:
URL: https://github.com/apache/gravitino/issues/11791
### What would you like to be improved?
Built-in IdP HTTP Basic authentication currently performs a **database
lookup on every authenticated request**, with no caching on the authentication
hot path.
For each request carrying `Authorization: Basic ...`, `BasicAuthenticator`
calls `IdpUserGroupManager.authenticate()`, which executes
`selectIdpUserWithGroups` — a SQL query against `idp_user_meta` joined with
`idp_user_group_rel` and `idp_group_meta` to fetch the password hash and group
memberships, followed by SHA3-512 password verification in memory.
Relevant code path:
- `plugins/idp-basic/.../BasicAuthenticator.java` → `authenticate()`
- `plugins/idp-basic/.../IdpUserGroupManager.java` → `authenticate()` →
`USER_SERVICE.getIdpUser()`
- `plugins/idp-basic/.../IdpUserMetaBaseSQLProvider.java` →
`selectIdpUserWithGroups`
**Why this matters**
1. **Per-request cost, not per-session** — Basic auth has no token/session
reuse like OAuth/JWT. Every API/IRC call repeats the IdP DB lookup + password
verification, even when the same user sends identical credentials repeatedly.
2. **Same path for User Auth and Group Auth** — Both authorization modes
call the same `authenticate()` method. Group-based authorization latency
(#11781) is dominated by downstream group→role resolution, but the Basic auth
step still adds a mandatory IdP query on every request regardless of auth mode.
3. **Amplified under load** — In performance tests (~13 RPS per client
stream, 30 concurrent users), this means hundreds of IdP SQL executions per
second cluster-wide. Local MySQL execution is sub-millisecond, but remote MySQL
adds network RTT per query; at high concurrency the cumulative overhead is
non-trivial.
4. **Discussion on #11781** — Maintainers noted that group membership may
already be cached elsewhere, but **password verification still queries the
database on every request**. This issue tracks caching specifically on the
Basic authentication path (user lookup, password hash fetch, and group
membership).
**Security note:** Any cache must invalidate promptly on password change,
user deletion, and group membership updates to avoid stale credentials or group
lists being accepted.
### How should we improve?
Potential approaches (open to discussion):
1. **Process-local cache (e.g., Caffeine) for IdP user metadata**
- Key: `(username)` or `(username, passwordHashVersion/current_version)`
- Value: password hash + group names (+ optional version sentinel from
`idp_user_meta.current_version`)
- Invalidate on: password update, user delete, group membership change
(via version bump or explicit eviction)
2. **Short-lived successful-auth cache**
- After a successful password verify, cache `(username, credential
fingerprint)` → `UserPrincipal` for a configurable TTL (e.g., seconds to
minutes)
- Must document trade-offs vs immediate revocation requirements
3. **Configuration knobs**
- TTL, max entries, enable/disable cache (default conservative for
security-sensitive deployments)
- Metrics: cache hit/miss, evictions
4. **Tests & docs**
- Unit/integration tests for cache hit, invalidation on password/group
change
- Document behavior in `docs/security/how-to-use-built-in-idp.md`
**Related:** #11781 (group-based authorization latency; complementary but
separate from authorization-side caching)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
