Guosmilesmile opened a new pull request, #18149: URL: https://github.com/apache/iceberg/pull/18149
Follow up https://github.com/apache/iceberg/pull/18144. Part of https://github.com/apache/iceberg/issues/18142 Add support for reloading the full cache of an Iceberg lookup join, instead of keeping the initial snapshot for the lifetime of the job. The reload is opt-in, so the existing behavior remains unchanged unless a reload option is configured. ### Changes - Support the standard Flink reload options: `lookup.full-cache.reload-strategy`, `periodic-reload.interval` / `.schedule-mode`, and `timed-reload.iso-time` / `.interval-in-days`. - Infer the strategy when it is not set: `PERIODIC` from `periodic-reload.interval`, `TIMED` from a timed reload option. No trigger is created when no option selects a strategy, so options like `periodic-reload.schedule-mode` do not enable reload on their own. - Use Flink's `PeriodicCacheReloadTrigger` and `TimedCacheReloadTrigger`. - Reload from the current snapshot and replace the existing cache only after the new one is fully loaded, so lookups always see a complete cache. - Skip reloads that find no new snapshot. This also makes the reload that `PERIODIC` and `TIMED` triggers issue on open a no-op for an already loaded cache. - Fail the next lookup when a reload fails, instead of serving the previous cache, because Flink's triggers do not propagate reload failures to the job. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
