Thanks Prashant for filing the PR. This is a solid start point for FGAC! The spec changes are straightforward and leave plenty of room for future extensibility. While the current Transform [1] isn’t quite sufficient for column masking, enhancements to Transform along with the upcoming UDF proposal should address that gap. For instance, we could introduce a "UDFTerm" once the UDF spec is finalized.
[1] https://github.com/polaris-catalog/polaris/blob/8d4cacb27c2a29353091b149e733b4fa7ce4ac53/spec/iceberg-rest-catalog-open-api.yaml#L2317-L2317 Yufei On Wed, Aug 20, 2025 at 2:59 PM Prashant Singh <prashant010...@gmail.com> wrote: > Robert, > > > It might not change this particular proposal > > having support for UDFs in Iceberg Expressions, which is a prerequisite > for this particular proposal > > Unfortunately, that's not my understanding of what community consensus was > based on community sync (here > <https://www.youtube.com/watch?v=RRyohCUDnME>) > I have put out a SPEC change proposal based on my understanding of what > community consensus is, as PR-13879 > <https://github.com/apache/iceberg/pull/13879>, I would request other > community members to chime here as well. > > As far as client server interaction with UDFs are concerned, based on my > understanding of discussion so far in UDF syncs, we want to send *all* > the versions / *dialects* that udf has from server to the client and the > client > picks the one which it wants, not the other way around. It is similar to > what we have for Iceberg View already, this should handle both your > questions. That being said if you have follow-up questions there is a > dedicated Sync for UDF discussions. > > Thank you, > Prashant Singh > > On Wed, Aug 20, 2025 at 2:48 AM Robert Stupp <sn...@snazy.de> wrote: > >> Prashant, >> >> be ensured that we are still and actively working into this. We are >> looking at this from multiple points of specific views but most >> importantly from a high level view: how all the things work seamlessly >> together. We are really looking forward to providing an update as soon >> as we are confident that the whole approach works. It might not change >> this particular proposal, but it is very important for a proper design >> to think about upstream and downstream impacts. >> >> In the meantime, as it's orthogonal to this particular effort, it >> would be great to get the ball rolling on having support for UDFs in >> Iceberg Expressions, which is a prerequisite for this particular >> proposal. We would be happy, if you could help getting that into >> Apache Iceberg. >> >> As we are speaking about Iceberg UDFs, and implicitly agreed that >> those become a prerequisite for this proposal, it would be very >> beneficial for clients (query engines) being able to leverage the >> following, which is where your help would be appreciated: >> 1. Allows clients to request an Iceberg UDF only in the current >> version, and only the representation (think: SQL dialect) the client >> needs. >> 2. Allow clients to request multiple Iceberg UDF representations in >> one request. Optionally only returning the currentversion and >> optionally restricted to the representation the client needs. >> >> Robert >> >> On Mon, Aug 18, 2025 at 6:05 PM Prashant Singh <prashant010...@gmail.com> >> wrote: >> > >> > Hi all, >> > >> > It was really nice to go through the proposal in the Iceberg catalog >> community sync (7/30)! >> > >> > As next steps, I was expecting the doc to be updated according to the >> community feedback, specifically: >> > >> > Using loadTable API to return the evaluated policies result. >> > >> > Aligning on Iceberg expressions as the path forward for row filters, >> with UDFs to handle dialect-specific logic. >> > >> > I noticed the doc hasn’t yet been updated with this feedback. I >> completely understand the authors might be busy, so I’d like to offer help >> to push this forward. >> > >> > We’ve discussed this topic in depth across several syncs, so perhaps we >> could even move ahead by opening a spec PR to IRC and continue iterating >> there. I’m happy to help put that out if it would help keep things moving. >> > >> > Downstream projects such as Apache Polaris are eagerly waiting for this >> feature, so it would be great to make progress here! >> > >> > Thanks, >> > Prashant Singh >> > >> > >> > On Wed, Jul 23, 2025 at 7:45 AM Robert Stupp <sn...@snazy.de> wrote: >> >> >> >> Hi all, >> >> >> >> Following up on the “Iceberg REST FGAC proposal” discussion [1], we >> >> are happy to share the more detailed proposal [2] to extend the Apache >> >> Iceberg REST specification to include a new API for retrieving >> >> fine-grained access control (FGAC) "protection instructions" >> >> (row-level filters and column transformations) from an Iceberg REST >> >> Catalog. >> >> >> >> The aim is to standardize how query engines obtain these instructions >> >> based on user identity, simplifying data protection enforcement. >> >> >> >> The proposal focuses solely on the new Iceberg REST API endpoint to >> >> retrieve protection instructions, intentionally omitting catalog >> >> specific policy management APIs. >> >> >> >> Having a truly interoperable way to represent the protection >> >> instructions for both row filters and column transformations is a huge >> >> benefit. This is why the support for Iceberg expressions is marked as >> >> mandatory in the proposal. We think that it is a fair option to allow >> >> people to use SQL expressions, not required by the proposal, to >> >> satisfy their needs, assuming they are okay to accept that not all >> >> catalogs or engines support SQL expressions or not all SQL >> >> conformance/dialects. >> >> >> >> Thanks to all of those who have helped review & contribute - JB >> >> Onofre, Prashant Singh, Russell Spitzer, Roy Hansson, & Kevin Liu. We >> >> are excited about the community support! >> >> >> >> Cheers, >> >> Robert, Laurent, Alex, Dmitri >> >> >> >> [1] https://lists.apache.org/thread/nfw1t0glfdfj1hwmzzzzwwyrfnq44yr5 >> >> [2] >> https://docs.google.com/document/d/108Y0E8XsZi91x-UY0_aHLEbmXDNmxmS5BnDjunEKvTM >> >
