Hi Ron, Thank you for the detailed and constructive feedback on the FLIP. These are all excellent points.
I have updated the FLIP's Confluence page, but I'd like to respond to each of your questions here as well. 1. On the Default Freshness Value and Configuration I agree that 3 minutes is a more sensible and safer default than 1 minute. To address this, I've updated the FLIP to include a new configuration option, materialized-table.default-freshness, with a default value of 3 min. This provides the best of both worlds: a safer default for everyone and the flexibility for administrators to configure it. 2. On the IntervalFreshness API Change (String to int) The FLIP now clarifies that we will use a non-breaking, "deprecate and add" strategy. We will keep the existing String getInterval() method but mark it as @Deprecated, and a new, type-safe int getIntervalInt() method will be added. This ensures existing code continues to compile and function while encouraging new development to use the safer method. I have updated the FLIP (Section 3.2) to make this explicit. 3. On the Scope and Discovery of MaterializedTableEnricher To keep the initial scope of this FLIP focused and deliverable, I've proposed that the framework will directly use the DefaultMaterializedTableEnricher for now. This provides a complete, working feature out of the box. The mechanism for discovering and configuring *custom* enrichers (e.g., via a service loader or environment configuration) is a crucial topic, but I believe it warrants its own dedicated discussion in a follow-up FLIP. This allows us to land this improvement pragmatically. 4. On RefreshContext You're right, the FLIP was missing the concrete design for this. I have updated the FLIP to include the full class definition for RefreshContext in the "Public Interfaces" section (now Section 3.6). It provides the enricher with the resolved freshness, logical refresh mode, and the freshness threshold from the configuration. Thanks again for helping to improve the proposal. Please let me know if these updates and the revised FLIP address your questions. Best, Ramin On Thu, Oct 9, 2025 at 5:34 AM Ron Liu <[email protected]> wrote: > Hi, Ramin > > Thanks for initiating this FLIP., This proposal can lower the threshold of > using Materialized Table, +1. > > I read through the FLIP, and I have the following questions: > > 1. Regarding the default freshness default value, you mentioned that it is > 1min, how is this value determined? In continuous mode, we will convert > freshness to checkpoint interval, in our internal best practice, the > checkpoint interval is usually 3min. > In addition, for data lake scenarios such as stream writing paimon, it is > more reasonable to set the checkpoint interval to 3min. In our internal > best practice, the checkpoint interval is usually 3min. > I think this default value needs to be discussed, or similar to > `materialized-table.refresh-mode.freshness-threshold`, we also provide an > option and provide a default value, which can be configured by the user. > > 2. You mentioned that to improve IntervalFreshness, change the current > string type to int for `interval`? The `getInterval()` public method is > already used by Paimon and other external systems, if we change the return > value type, it will cause incompatible behavior. > > 3. About `MaterializedTableEnricher` interface, how to pass the > user-defined implementation to the framework, i.e., how the framework > perceives it? Also, does the scope of this interface support customization > at the table level, or the Catalog level? Or Service level? > > 4. Regarding RefreshContext, can you give a concrete design of the > interface and what information it will provide to > MaterializedTableEnricher? > > > Best, > Ron > > Ramin Gharib <[email protected]> 于2025年10月6日周一 20:19写道： > > > Hi everyone, > > > > I would like to start a discussion on FLIP-551 [1], which proposes making > > the FRESHNESS clause optional for Materialized Tables. > > > > The goal is to simplify the user experience for common streaming cases > and > > to enable more powerful, platform-level default logic. > > > > The proposal achieves this by: > > > > 1. > > > > Making the FRESHNESS DDL clause optional, falling back to a sensible > > default of 1 minute. > > 2. > > > > Introducing a new, MaterializedTableEnricher interface for custom > logic > > to resolve the final Freshness and RefreshMode into the > > ResolvedCatalogMaterializedTable when they are omitted. > > > > This change reduces boilerplate for users while providing a clean > extension > > point for advanced use cases. The complete architectural details and API > > are in the FLIP. > > > > I'm looking forward to hearing your feedback. > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-551%3A+Make+FRESHNESS+Optional+for+Materialized+Tables > > > > Thanks, > > > > Ramin > > >
