[
https://issues.apache.org/jira/browse/SPARK-59726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huaxin Gao updated SPARK-59726:
-------------------------------
Labels: SPIP (was: )
> SPIP: Pluggable Row-Filter and Column-Mask Enforcement in Catalyst
> ------------------------------------------------------------------
>
> Key: SPARK-59726
> URL: https://issues.apache.org/jira/browse/SPARK-59726
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Huaxin Gao
> Priority: Major
> Labels: SPIP
>
> External catalogs and governance systems need to enforce fine-grained access
> control, row-level filtering and column masking, on data read through Spark.
> Today Spark has no supported API for a data source or catalog to attach such
> policies to a scan and have them enforced correctly in the query plan.
> Integrators must inject Catalyst rules through SparkSessionExtensions and use
> package-private Catalyst internals, which is brittle across versions and
> unsafe: a masking projection added naively can be removed or collapsed by the
> optimizer, silently returning unmasked data.
> This SPIP proposes a stable, public, optimizer-safe DataSource V2 API for a
> table to declare a read access policy (readable columns, a row filter, and a
> set of column masks) that Spark enforces with fail-closed semantics: the row
> filter evaluates on original values before masking, the masks cannot be
> optimized away, and anything Spark cannot resolve fails the read rather than
> returning unprotected data.
> SPIP document:
> https://docs.google.com/document/d/1hYEHORjHUnFzSoBY2DeOPILVJyu6iWXBusYwpF9FQJM/edit?tab=t.0
> Discussion thread: <to be added >
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
