FrankChen021 opened a new issue, #20847:
URL: https://github.com/apache/druid/issues/20847
### Motivation
When a SQL query fails authorization, Druid returns:
```json
{"error":"druidException","errorCode":"forbidden","persona":"USER","category":"FORBIDDEN","errorMessage":"Unauthorized"}
```
The user cannot tell which permission is missing. An MSQ ingestion such as
```sql
SET finalizeAggregations = FALSE;
REPLACE INTO "kttm" OVERWRITE ALL
SELECT ... FROM TABLE(EXTERN('{"type":"http",...}', ...)) ...
```
is checked against several kinds of resources, and failing any one of them
produces the same message:
| Resource type | Name | Action | Required by |
|---|---|---|---|
| `DATASOURCE` | `kttm` | WRITE | `INSERT` / `REPLACE INTO kttm` |
| `DATASOURCE` | each table read | READ | `FROM` / `JOIN` |
| `VIEW` | each view read | READ | `FROM view.x` (underlying tables are not
checked) |
| `EXTERNAL` | `EXTERNAL`, or the input source type (e.g. `http`) when
`druid.auth.enableInputSourceSecurity=true` | READ | `EXTERN(...)` and other
external table functions |
| `EXTERNAL` | the export destination type | WRITE | `INSERT INTO
EXTERN(...)` exports (`IngestHandler`) |
| `QUERY_CONTEXT` | each secured context key | WRITE | when
`druid.auth.authorizeQueryContextParams=true`; secured keys are the keys the
user set, minus Druid's always-allowed keys and
`druid.auth.unsecuredContextKeys`, intersected with
`druid.auth.securedContextKeys` if that is set
(`AuthConfig.contextKeysToAuthorize`) |
Users must ask an operator to read Broker logs, or bisect the query by trial
and error. Three gaps cause this:
1. **The denied resource is dropped.**
`AuthorizationUtils.authorizeAllResourceActions()` knows which `ResourceAction`
was denied (it reports it in the `auth/forbidden` metric, #19552) but returns
only `Access.getMessage()`. `BasicRoleBasedAuthorizer` denies with an empty
reason, so that is `"Unauthorized"`.
2. **The MSQ endpoints replace the message.** `SqlStatementResource` (4
`catch (ForbiddenException e)` sites) and `SqlTaskResource` return the constant
`"Unauthorized"` whatever the exception says.
3. **Users cannot inspect their own permissions.** `Authorizer` exposes only
`authorize(AuthenticationResult, Resource, Action)`. The basic-security user
API
(`/druid-ext/basic-security/authorization/db/{authorizer}/users/{user}?full`)
is an admin API.
### Proposed changes
#### Part 1: Name the denied resource in SQL 403 messages
Before:
```
Unauthorized
```
After:
```
Unauthorized: user[alice] has no READ permission on resource type[EXTERNAL]
name[http] in authorizer[basic]. Ask your administrator to grant this
permission to you or your roles.
```
- **Where it applies:** SQL only, meaning `/druid/v2/sql`, the MSQ APIs
(`/druid/v2/sql/statements` submission, `/druid/v2/sql/task`), Dart and JDBC.
Every resource checked there comes from the query text or the query context, so
the message names nothing the caller did not send. Non-SQL REST endpoints keep
returning `Unauthorized`, because some of them check resources the caller did
not name (for example `TaskResourceFilter` authorizes the datasource of a task
looked up by ID).
- **Unchanged:** `errorCode` (`forbidden`), `category`, `persona` and HTTP
status 403.
- **`druid.server.http.errorResponseTransform` is respected:** where the
configured strategy would hide the message, the response is the bare
`Unauthorized` as before.
#### Part 2: `sys.grants`
**SPI.** Add one default method to `Authorizer`:
```java
/**
* Returns the permissions held by the identity in authenticationResult, or
Optional.empty() if this
* authorizer cannot enumerate permissions. Must not return null.
*/
default Optional<List<Grant>> listGrants(AuthenticationResult
authenticationResult)
{
return Optional.empty();
}
```
`Grant` is a new value class in `org.apache.druid.server.security` holding
`resourceType`, `resourceName` and `action`. They map to the same-named table
columns below; `authorizer` and `identity` come from the caller's
`AuthenticationResult`.
**Table.** `sys.grants`, evaluated on the Broker, returns one row per
permission held by the calling identity. Columns follow `SystemSchema`
conventions (snake_case, `STRING`/`LONG` types):
| Column | Type | Meaning |
|---|---|---|
| `authorizer` | STRING | Name of the caller's authorizer |
| `identity` | STRING | The caller's identity. Always the same value;
included so rows are self-describing when exported or joined. |
| `resource_type` | STRING | One of `ResourceType.knownTypes()`; never a
wildcard |
| `resource_name` | STRING | The name or pattern as stored by the
authorizer, in that authorizer's syntax. For both authorizers in this proposal
it is a Java regex that must match the whole resource name. |
| `action` | STRING | `READ` or `WRITE` |
Rules:
- **Caller only.** Rows come from the `AuthenticationResult` in the query's
data context (`DATA_CTX_AUTHENTICATION_RESULT`, as `SystemSchema` already does
for row filtering). There is no way to read another identity's grants through
this table; admins keep the basic-security API for that.
- **Unsupported authorizers fail.** If `listGrants` returns
`Optional.empty()`, the query fails with `DruidException` persona `USER`,
category `UNSUPPORTED`: `Authorizer [<name>] does not support listing grants`.
It must not return zero rows, which would read as "you hold no permissions".
- **Policy restrictions are not shown.** A row means the action is allowed;
it may still be subject to a row-level policy (for example one configured on
`AllowAllAuthorizer`, or returned by a custom authorizer).
- **No duplicate rows.** When several roles grant the same permission, it
appears once. The table does not say which role a permission comes from.
- **Access to the table** follows the existing system-table rules. With
`druid.sql.planner.authorizeSystemTablesDirectly=true`, it needs READ on
`SYSTEM_TABLE` `grants`; otherwise no extra permission.
**Implementations in this proposal:**
- **`BasicRoleBasedAuthorizer`:** the permissions of every role returned by
`roleProvider.getRoles(name, authenticationResult)`, which includes roles
assigned directly and roles from LDAP group mappings. One row per
`BasicAuthorizerPermission`. Reads only the in-memory role cache.
- **`AllowAllAuthorizer`:** one row for each type in
`ResourceType.knownTypes()` × each `Action`, with `resource_name = '.*'`.
- **`druid-ranger-security` and third-party authorizers:** inherit the
default and report `UNSUPPORTED`.
**Sample rows.** User `alice` on a basic-security authorizer named `basic`,
holding two roles: `analyst` (read every datasource, read cluster state) and
`kttm_ingest` (write `kttm*` datasources, read HTTP input sources, set one
secured context key). `SELECT * FROM sys.grants` returns:
| authorizer | identity | resource_type | resource_name | action |
|---|---|---|---|---|
| basic | alice | DATASOURCE | .* | READ |
| basic | alice | STATE | STATE | READ |
| basic | alice | DATASOURCE | kttm.* | WRITE |
| basic | alice | EXTERNAL | http | READ |
| basic | alice | QUERY_CONTEXT | finalizeAggregations | WRITE |
For the MSQ query in Motivation, these rows show `alice` holds WRITE on
`kttm`, READ on `EXTERNAL` `http`, and the `finalizeAggregations` context key.
If `druid.auth.enableInputSourceSecurity` were false, the query would need READ
on `EXTERNAL` named `EXTERNAL`, which no row grants.
The same query on a cluster using `AllowAllAuthorizer` (authorizer and
identity `allowAll`) returns one row per known resource type and action (rows
abbreviated):
| authorizer | identity | resource_type | resource_name | action |
|---|---|---|---|---|
| allowAll | allowAll | DATASOURCE | .* | READ |
| allowAll | allowAll | DATASOURCE | .* | WRITE |
| allowAll | allowAll | EXTERNAL | .* | READ |
| … | | | | |
Filtering works like any other table:
```sql
SELECT resource_type, resource_name, action
FROM sys.grants
WHERE resource_type = 'EXTERNAL'
```
`resource_name` holds patterns, so `WHERE resource_name = 'kttm'` does not
find a `.*` grant. To ask "can I write `kttm`?", match the pattern the way the
authorizer does. For basic-security and allow-all, anchor the regex, because
`REGEXP_LIKE` finds a match anywhere in the string:
```sql
SELECT resource_name
FROM sys.grants
WHERE resource_type = 'DATASOURCE'
AND action = 'WRITE'
AND REGEXP_LIKE('kttm', '^(?:' || resource_name || ')$')
```
### Rationale
- **Name the resource only on the SQL path.** The shared authorization code
keeps the authorizer's message unchanged and records the denied resource
separately; only the SQL layer, where every checked resource comes from the
query, puts it into the message. Non-SQL callers stay unchanged.
- **Part 1 answers "why was I denied"; `sys.grants` answers "what do I
hold".** Together they cover the self-service need without a separate
permission-check function. Such a function (like PostgreSQL's
`has_table_privilege` or SQL Server's `HAS_PERMS_BY_NAME`) is not proposed.
- **System table instead of `SHOW GRANTS` syntax.** No parser grammar
change; composes with `WHERE`/`ORDER BY`; works the same from the web console,
JDBC and the SQL API, like `sys.tasks` and `sys.servers`.
- **Comparison with other databases.** PostgreSQL
(`information_schema.table_privileges`), MySQL (`TABLE_PRIVILEGES`), ClickHouse
(`system.grants`), Snowflake (`SHOW GRANTS`) and StarRocks
(`sys.grants_to_users`) all include a grantee, which `identity` covers. Most
also distinguish a grant to a user from a grant through a role; this proposal
does not. `grant_option`/`is_grantable` is omitted because Druid SQL has no
`GRANT`. Only ClickHouse models wildcards and partial revokes. Both authorizers
here use one pattern syntax, so no syntax column is needed yet (see Future
work), and neither produces deny grants. A separate role-membership table
(ClickHouse `system.role_grants`) is not needed because basic-security roles do
not nest.
- **Rejected: listing a query's required resources via EXPLAIN.** `EXPLAIN
PLAN FOR` reports resources but is authorized like the query itself, so a user
lacking the permission gets the same 403.
### Operational impact
- **SQL error text changes.** Clients that match the exact string
`"Unauthorized"` on SQL responses must check `errorCode == "forbidden"` or HTTP
403 instead. Release notes will say so. Non-SQL endpoints are unchanged.
- **New public API surface:** `sys.grants`, `Authorizer.listGrants`, `Grant`.
- **No new configuration, no metadata-store change, no rolling-upgrade
ordering.** Authorizer authors need no action.
### Future work
- `SHOW GRANTS` as a parser shortcut for `SELECT * FROM sys.grants`.
- `listGrants` for `druid-ranger-security`, together with a
`resource_name_syntax` column (e.g. `JAVA_REGEX_FULL_MATCH`, `WILDCARD`,
`EXACT`), because Ranger patterns use `*`/`?` wildcards rather than Java regex.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
