[
https://issues.apache.org/jira/browse/FLINK-38597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rosa Seohwa Kang updated FLINK-38597:
-------------------------------------
Description:
In multi-tenant deployments with session clusters, the SQL Gateway currently
lacks a built-in request-level audit trail.
Operators need to attribute queries to users, track access to datasets, and
meet compliance requirements (privacy, retention, provenance).
A standardized, configurable audit log would avoid bespoke proxies/sidecars and
make governance first-class in Flink.
Proposed solution:
* Add a handler to capture REST requests in RestServerEndpoint (when enabled
via Flink configuration).
* Introduce a log writer for batching log lines, rotates file by size,
periodic flush.
* Supports gs:// via Flink FileSystem; filters by HTTP methods, endpoints,
headers; logs body with truncation/sanitization.
* When enabled, Gateway emits structured request logs and can persist to GCS.
* Logging is configurable, low-overhead, and shuts down cleanly.
** A prototype is currently being worked on. A PR will be shared soon.
was:
In multi-tenant deployments with session clusters, the SQL Gateway currently
lacks a built-in request-level audit trail.
Operators need to attribute queries to users, track access to datasets, and
meet compliance requirements (privacy, retention, provenance).
A standardized, configurable audit log would avoid bespoke proxies/sidecars and
make governance first-class in Flink.
Proposed solution:
* Add a handler to capture REST requests in RestServerEndpoint (when enabled
via Flink configuration).
* Introduce a log writer for batching log lines, rotates file by size,
periodic flush.
* Supports gs:// via Flink FileSystem; filters by HTTP methods, endpoints,
headers; logs body with truncation/sanitization.
* When enabled, Gateway emits structured request logs and can persist to GCS.
* Logging is configurable, low-overhead, and shuts down cleanly.
* A prototype is currently being worked on. A PR will be shared soon.
> Add request audit logging with pluggable sinks (GCS support)
> ------------------------------------------------------------
>
> Key: FLINK-38597
> URL: https://issues.apache.org/jira/browse/FLINK-38597
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / REST
> Affects Versions: 2.1.1
> Reporter: Rosa Seohwa Kang
> Priority: Minor
> Fix For: 2.1.2
>
>
> In multi-tenant deployments with session clusters, the SQL Gateway currently
> lacks a built-in request-level audit trail.
> Operators need to attribute queries to users, track access to datasets, and
> meet compliance requirements (privacy, retention, provenance).
> A standardized, configurable audit log would avoid bespoke proxies/sidecars
> and make governance first-class in Flink.
>
> Proposed solution:
> * Add a handler to capture REST requests in RestServerEndpoint (when enabled
> via Flink configuration).
> * Introduce a log writer for batching log lines, rotates file by size,
> periodic flush.
> * Supports gs:// via Flink FileSystem; filters by HTTP methods, endpoints,
> headers; logs body with truncation/sanitization.
> * When enabled, Gateway emits structured request logs and can persist to GCS.
> * Logging is configurable, low-overhead, and shuts down cleanly.
> ** A prototype is currently being worked on. A PR will be shared soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
