raminqaf opened a new pull request, #27132:
URL: https://github.com/apache/flink/pull/27132
## What is the purpose of the change
This pull request makes the `FRESHNESS` clause optional when creating
materialized tables. When freshness is not specified, the system uses
configurable defaults based on the refresh mode:
`materialized-table.default-freshness.continuous` (default: 3 minutes) for
CONTINUOUS mode, or `materialized-table.default-freshness.full` (default: 1
hour) for FULL mode. This provides users with more flexibility and allows
catalogs to implement custom freshness and refresh mode determination logic.
## Brief change log
- Introduced `MaterializedTableEnricher` interface for pluggable freshness
and refresh mode resolution logic
- Implemented `DefaultMaterializedTableEnricher` with threshold-based
refresh mode determination
- Added two configuration options:
- `materialized-table.default-freshness.continuous` (default: 3 minutes)
- `materialized-table.default-freshness.full` (default: 1 hour)
- Made `FRESHNESS` clause optional in `CREATE MATERIALIZED TABLE` syntax
- Updated `SqlCreateMaterializedTableConverter` to use the enricher for
determining freshness and refresh mode
- Refactored refresh mode determination from being computed once in
converter to being computed on-demand via interface default methods
- Updated documentation to reflect optional freshness and mode-specific
defaults
- Maintained backward compatibility with existing catalog implementations
## Verifying this change
This change is already covered by existing tests, such as:
- `SqlMaterializedTableNodeToOperationConverterTest.testFullRefreshMode()`
- validates refresh mode determination
-
`SqlMaterializedTableNodeToOperationConverterTest.testContinuousRefreshMode()`
- validates continuous mode behavior
- Existing materialized table integration tests verify end-to-end
functionality
Additionally, this change maintains backward compatibility with existing
materialized table implementations through default interface methods in
`CatalogMaterializedTable`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes (CatalogMaterializedTable interface)
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs / JavaDocs
- Updated materialized table documentation (overview.md and
statements.md)
- Added JavaDocs for new interfaces and classes
- Updated configuration documentation to include new options
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
