Now I realize it wasn't mentioned if it is a SPI (something to extend the codebase with) or a REST API (webhook). A public REST API for notifications is cause of concern for me as it introduces some new complexity to deal with availability issues.
So what type of APIs are we talking about? Laurent On Tue, Oct 29, 2024 at 11:13 AM Russell Spitzer <russell.spit...@gmail.com> wrote: > I know we've had a lot of discussions about this and I think we were hoping > to get this into the Iceberg REST API as well. I think we should try to > push the API there but I have no problem having it only within the Polaris > spec if we can't get that in. I think it has a lot of overlap with the > things we have been talking about in Yufei's Table Maintenance proposal as > well. > > On Tue, Oct 29, 2024 at 10:58 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > > > Hi folks, > > > > I would like to discuss a new proposal for a polling notification API. > > The proposal actually contains two "layers": > > - Entities change notification API (optionally with Java client to > > poll): a client can receive notifications when an event happens on an > > entity (containing the entity type, like table, view, namespace, > > catalog, ...). Events can be entities created, modified, deleted. The > > use case is basically to have both maintenance anticipation, > > incremental index update, actions triggering. > > - Entities change history: a client can get the list of all entities > > changes in the past N duration (hours, minutes, seconds). It could be > > an async or sync operation. The use case is about reconciliation and > > auditing. > > > > Thoughts ? > > > > If there's no objections, I will work on a proposal document (with > > much more details) that I will share on the dev mailing list (and also > > create a GitHub Issue with "proposal" tag). > > > > Regards > > JB > > >
