GitHub user pavetheroad41 added a comment to the discussion: [Ideas] data encrypt or privacy infomation protection
Thanks for the detailed writeup, reframing this as dynamic data masking (transform-at-egress) rather than TDE/pgcrypto clarifies the scope a lot, and these are the right questions to be asking at this stage. On your open implementation questions: ### 1. Catalog invalidation for policy metadata Cloudberry/Postgres already has most of the plumbing for this: DDL on the coordinator is dispatched to segments via the existing QD→QE dispatch path, and syscache/relcache invalidation runs on `SharedInvalidationMessage` (sinvaladt.c). A new `pg_privacy_policy` catalog could piggyback on that mechanism, new invalidation message type, bump a generation counter, backends lazily refresh their L1 cache. That's close to what you sketched, so it likely doesn't need new infrastructure. ### 2. Session GUC vs. auth hook A custom GUC (`privacy.token`) is workable, but I'd validate it closer to connection time (e.g. `ClientAuthentication_hook`) rather than allowing arbitrary mid-session `SET`, and explicitly strip it from `pg_stat_activity.query` and audit-log text, not just `SHOW ALL`. Since the JWT is a bearer credential, leakage risk is about replay within its validity window; short expiry + audience binding (as you proposed) is the right mitigation, but the redaction needs to be more aggressive than a GUC flag provides. ### 3. Client-output boundary - Planner vs. GPORCA Postgres already gives you most of this distinction: regardless of which planner produced the plan, execution ends by pushing tuples through a `DestReceiver`. Plain `SELECT` uses `DestRemote`; `CTAS`/`SELECT` INTO use `DestIntoRel` - already a different receiver. That's nearly the exact dichotomy you want, and it's planner-agnostic, so you'd hook at the receiver level rather than in ORCA or the PG planner. Two gaps to flag: `COPY ... TO STDOUT` doesn't go through `DestReceiver`, it's a separate path in `copy.c` and needs its own hook. `RETURNING` and cursor `FETCH` do use the normal tuple destination, so those should fall out for free. ### 4. Pushing the projection to segments Feasible if it's inserted as a post-optimization wrapper - just below the final Gather Motion, after planning completes - rather than something the optimizer costs, so it doesn't perturb join/agg/sort decisions. Worth noting as precedent: RLS + GPORCA has had rough edges historically (ORCA falls back to the PG planner in some RLS cases) - a useful cautionary example for how an ORCA-blind feature like this could get stuck. ### 5. Extension vs. core Given the `COPY` gap and the need for cross-node cache invalidation, I don't think this stays cleanly in extension territory. A reasonable path: prototype as an extension covering plain `SELECT`/`RETURNING` only (validates policy DDL + caching + JWT auth end-to-end), then upstream once proven, since `COPY` interception and ORCA-safe plan injection will need to land in core regardless. Given the scope, I'd second Lirong's suggestion of a short design doc - the catalog schema, cache invalidation protocol, and `DestReceiver` hook points would be the key pieces to nail down before writing code. GitHub link: https://github.com/apache/cloudberry/discussions/1943#discussioncomment-18272341 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
