I know we've had a lot of discussions about this and I think we were hoping to get this into the Iceberg REST API as well. I think we should try to push the API there but I have no problem having it only within the Polaris spec if we can't get that in. I think it has a lot of overlap with the things we have been talking about in Yufei's Table Maintenance proposal as well.
On Tue, Oct 29, 2024 at 10:58 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi folks, > > I would like to discuss a new proposal for a polling notification API. > The proposal actually contains two "layers": > - Entities change notification API (optionally with Java client to > poll): a client can receive notifications when an event happens on an > entity (containing the entity type, like table, view, namespace, > catalog, ...). Events can be entities created, modified, deleted. The > use case is basically to have both maintenance anticipation, > incremental index update, actions triggering. > - Entities change history: a client can get the list of all entities > changes in the past N duration (hours, minutes, seconds). It could be > an async or sync operation. The use case is about reconciliation and > auditing. > > Thoughts ? > > If there's no objections, I will work on a proposal document (with > much more details) that I will share on the dev mailing list (and also > create a GitHub Issue with "proposal" tag). > > Regards > JB >
