Hi all, Got a message that the formatting was hard to comprehend, so I’ve reformatted the below email. There is no new information in this email, but feel free to read this instead! —————
Hi all, Thanks to those who could make the Polaris Community Sync regarding the Events API on 2025-08-18. Below are the notes (and my further thoughts regarding the discussion topics) from the meeting. As discussed (without any expressed disagreement), there will be: - A one-week window (ending on 2025-08-25) to express any blocking concerns with any of the approaches or proposals. - A further one-week window (ending on 2025-09-01) to suggest approaches to resolve these concerns. Not responding within these time windows will be considered “reached consensus.” Please ensure any responses clearly indicate which proposal/PR the comments or feedback are for – or feel free to comment/review on the PRs directly. Thanks to the whole community for helping to move the Events feature forward! ————— Proposal #1 – PR #1904 <https://github.com/apache/polaris/pull/1904>: Additional Instrumentation for New Events As discussed at the sync, there are no notable changes to the Events and Event Listener interfaces within this PR - this proposal solely adds new instrumentation for events using the existing interfaces and emulates current implementations of said interfaces. Concerns & Responses: - Too many files in this PR: I have committed to refactoring all Event records into a shared Java class. I will push this change with the next revision. - Too much code in business logic: I can commit to refactoring PolarisServiceImpl.java into a delegator pattern (see Appendix 1). This allows future support for onError… methods if error handling is desired. To be clear, error handling was not a stated use-case of the Events API - but looks to be a straightforward extension. - “Lack of functionality” for correlating before/after events: - This is really an argument for a new Events API, which should likely require its own proposal. This proposal is simply an extension of the currently-implemented API. - However, the current design remains compatible with a future requirement for a stateful event handler. I’ve outlined a small backwards-compatible change in Appendix 2 showing this. I’m happy to field questions on this proposed change in an unrelated thread to PR #1904, either on Slack or in a different mail thread. Thanks to Russell Spitzer for helping identify some of these solutions! ————— Proposal #2 – PR #1844 <https://github.com/apache/polaris/pull/1844>: Add Polaris Events to Persistence • Not thoroughly discussed at the sync due to time constraints. • Mentioned briefly as a sample implementation of the Events Listener interface. • No unaddressed concerns remain. • I am surfacing this as a way for a “last call” for feedback before this gets merged. ————— Proposal #3 – PR #1965 <https://github.com/apache/polaris/pull/1965>: Add Event Listener Implementation using AWS CloudWatch as Event Sink • Not discussed at the sync due to time constraints. • The eventual goal is to establish a repeatable pattern for other cloud logging products. • A few minor comments remain on the PR; I’ll address these shortly. • There are plans to merge this PR following that resolution. • This is an also being surfaced for a “last call” for feedback. ————— Regards, Adnan Hemani ————— Appendix 1: EventingPolarisServiceImpl.java public class EventingPolarisServiceImpl implements PolarisCatalogsApiSerivce, PolarisPrincipalsApiService, PolarisPrincipalsRolesApiService { private final PolarisServiceImpl delegate; private final PolarisEventListener listener; public … createCatalog(…) { listener.onBeforeCreateCatalog(…); try { delegate.createCatalog(…); listener.onAfterCreateCatalog(…); } catch (Exception e) { listener.onExceptionCreateCatalog(…); } } } ————— Appendix 2: EventingPolarisServiceImpl.java public class EventingPolarisServiceImpl implements PolarisCatalogsApiSerivce, PolarisPrincipalsApiService, PolarisPrincipalsRolesApiService { private final PolarisServiceImpl delegate; private final PolarisEventListener listener; public … createCatalog(…) { String eventId = listener.onBeforeCreateCatalog(…); try { delegate.createCatalog(…); listener.onAfterCreateCatalog(<event>, eventId); } catch (Exception e) { listener.onExceptionCreateCatalog(<event>, eventId); } } } PolarisEventListener.java: // The Event Listener implementation generates and returns the eventId. public String onBeforeCreateCatalog(<event>) {} // Uses eventId if desired. public void onAfterCreateCatalog(<event>, String eventId) {} // Backwards compatibility: send `null` as eventId if unused. // Can also mark as Deprecated for easier maintenance. public void onAfterCreateCatalog(<event>) { onAfterCreateCatalog(<events>, null); } // Not yet created, so no backwards-compatibility needed. public void onExceptionCreateCatalog(<event>, String eventId) {} > On Aug 19, 2025, at 12:08 AM, Adnan Hemani <adnan.hem...@snowflake.com> wrote: > > Hi all, > > Thanks to those who could make the Polaris Community Sync regarding the > Events API on 2025-08-18. Below are the notes (and my further thoughts > regarding the discussion topics) from the meeting. As discussed (without any > expressed disagreement), there will be a one-week window (ending on > 2025-08-25) to express any blocking concerns with any of the approaches or > proposals, and a further one week window to then suggest any approaches to > resolve these concerns (ending on 2025-09-01). Not responding within these > time windows will be considered “reached consensus.” Please ensure any > responses clearly indicate which proposal/PR the comments or feedback are for > - or feel free to comment/review on the PRs directly! > > Thanks to the whole community for helping to move the Events feature forward! > > Proposal #1 - PR #1904 <https://github.com/apache/polaris/pull/1904>: > Additional Instrumentation for New Events > > As discussed at the sync, there are no notable changes to the Events and > Event Listener interfaces within this PR - this proposal solely adds new > instrumentation for events using the existing interfaces and emulates current > implementations of said interfaces. > > Regarding concerns of adding too many files within this PR: I have committed > to refactoring all Event records into a shared Java class. I will push this > change with the next revision. > Regarding concerns of adding too much code within the business logic: I can > commit to refactoring `PolarisServiceImpl.java` to create a delegator pattern > where each method would emit events in a separate Java class and delegate > back to `PolarisServiceImpl.java`. This could look something like Appendix 1. > This pattern should satisfy the ability for someone (in a future PR) to > implement `onError…` methods, in case we would like to do error handling > using the Events API. To be clear, error handling was not a stated use-case > of the Events API - but looks to be a straightforward extension. > Regarding concerns of the “lack of functionality” to correlate before/after > events together: > As stated at the sync, this is a false statement. There is a way to do this > already - but it is up to the Event Listener implementation to make this > happen. > All to-date-publicized alternatives would also need to handle the logic of > correlating before/after events within the Event Listener implementation, and > so I cannot consider them as “strictly better”. The alternatives would also > change the easy-to-implement interfaces to more complex ones, which is a > significant drawback to this feature. Any interface that requires user > implementation should always be as simple as possible. > There is not any concrete/known use case for this requirement, nor was it > considered an original goal of the Events API. Given all the above, it is > currently hard for me to understand this as a blocker for merging further > event instrumentation with the current set of interfaces. > On a related note, any major changes to the Event interfaces would already > need to be considered as “breaking changes,” as it is already released and > public within our Polaris 1.0.0 release. > There is also a potential solution to pass in the “before” event ID with a > small backwards-compatible change to the Events Listener interface (Appendix > 2). I’m happy to field questions on this proposed change in an unrelated > thread to PR #1904, either on Slack or in a different mail thread. > > Many thanks to Russell Spitzer for helping to identify some of the solutions > listed above! > > — > > Proposal #2 - PR #1844 <https://github.com/apache/polaris/pull/1844> Add > Polaris Events to Persistence > > This was not thoroughly discussed during the Community Sync due to lack of > time, but was touched upon briefly whenever it was relevant as a new sample > implementation of the Events Listener interface. > > There are currently no unaddressed concerns or comments on this PR. I am > surfacing this as a way for a “last call” for feedback before this gets > merged. > > — > > Proposal #3 - PR #1965 <https://github.com/apache/polaris/pull/1965> Add > Event Listener Implementation using AWS CloudWatch as a event sink > > This was not mentioned during the community sync due to lack of time as well. > The goal is hopefully to repeat the patterns in this PR for other cloud > service provider logging products. > > There are a few minor comments remaining on this PR, which I will address in > the upcoming days. There are plans to merge this PR following that > resolution. This is an also being surfaced for a “last call” for feedback. > > Regards, > Adnan Hemani > > — > > Appendix 1: > > EventingPolarisServiceImpl.java: > ``` > public class EventingPolarisServiceImpl implements PolarisCatalogsApiSerivce, > PolarisPrincipalsApiService, PolarisPrincipalsRolesApiService { > private final PolarisServiceImpl delegate; > private final PolarisEventListener listener; > > public … createCatalog(…) { > listener.onBeforeCreateCatalog(…); > try { > delegate.createCatalog(…); > listener.onAfterCreateCatalog(…); > } catch (Exception e) { > listener.onExceptionCreateCatalog(…); > } > } > } > ``` > > Appendix 2: > > EventingPolarisServiceImpl.java: > ``` > public class EventingPolarisServiceImpl implements PolarisCatalogsApiSerivce, > PolarisPrincipalsApiService, PolarisPrincipalsRolesApiService { > private final PolarisServiceImpl delegate; > private final PolarisEventListener listener; > > public … createCatalog(…) { > String eventId = listener.onBeforeCreateCatalog(…); > try { > delegate.createCatalog(…); > listener.onAfterCreateCatalog(<event>, eventId); > } catch (Exception e) { > listener.onExceptionCreateCatalog(<event>, eventId); > } > } > ``` > > PolarisEventListener.java: > ``` > … > // The Event Listener implementation would then be in charge of generating > the eventId and returning it… > public String onBeforeCreateCatalog(…) {} > > // …as well as using it further (if it would like). > public void onAfterCreateCatalog(<event>, String eventId) {} > > // We can achieve backwards compatibility through sending in `null` as the > event ID. > // Event Listener implementations would need to handle this case. > // Additionally, we can mark this as Deprecated, if it makes for easier > maintenance > public void onAfterCreateCatalog(<event>) { > onAfterCreateCatalog(<events>, null); > } > > // No need to handle backwards-compatibility here as this is not yet created > and is still a proposal. > public void onExceptionCreateCatalog(<event>, String eventId) {} > … > ```
