sd4324530 opened a new pull request, #3521:
URL: https://github.com/apache/fluss/pull/3521
### Purpose
Linked issue: close #3520
This PR adds Flink 2.3 engine support to Apache Fluss by introducing a new
`fluss-flink-2.3` module. To accommodate the API changes in
`CatalogMaterializedTable` / `IntervalFreshness` introduced in Flink 2.3,
corresponding version adapters are introduced in the common module so that
cross-version connector code can compile against multiple Flink majors.
### Brief change log
**1. New `fluss-flink-2.3` module**
- Added `fluss-flink-2.3` as a sub-module in `fluss-flink/pom.xml` to
provide connector support for Flink 2.3-based deployments.
- The module mirrors the structure of `fluss-flink-2.2` and ships with
complete source and test directories. Version-specific adapters (e.g.,
`MultipleParameterToolAdapter`, `SchemaAdapter`, `SinkAdapter`,
`TypeInformationAdapter`) follow the existing pattern and are provided
alongside the module.
**2. New version adapters in the common module**
To handle the API changes of `CatalogMaterializedTable` /
`IntervalFreshness` in Flink 2.3, the following adapters are introduced under
`fluss-flink-common/src/main/java/org/apache/fluss/flink/adapter/`:
- `CatalogMaterializedTableAdapter`: wraps
`CatalogMaterializedTable.Builder` to abstract away differences introduced in
Flink 2.3 (such as the new `originalQuery` / `expandedQuery` fields), so the
shared common code does not depend on a specific Flink version.
- `IntervalFreshnessAdapter`: promotes the logic previously living in the
test-only `ResolvedCatalogMaterializedTableAdapter` into public API, providing
a unified way to parse and serialize `IntervalFreshness.TimeUnit` (whose enum
structure has been adjusted in Flink 2.3).
**3. Configuration and serialization compatibility**
- `FlinkConnectorOptions.MATERIALIZED_TABLE_INTERVAL_FRESHNESS_TIME_UNIT` is
changed from `enumType(IntervalFreshness.TimeUnit.class)` to `stringType()`,
avoiding a hard dependency on a Flink-version-specific enum class in the common
module; concrete enum parsing is delegated to `IntervalFreshnessAdapter`.
- `FlinkConversions` is updated to use the new adapters for
materialization-table serialization/deserialization, ensuring consistent
semantics across versions.
**4. Back-port to fluss-flink-2.2**
`CatalogMaterializedTableAdapter` is also added to `fluss-flink-2.2`, and
`Flink22CatalogTest` is updated accordingly, so that the 2.2 module continues
to share the same common code after the new adapter is introduced.
**5. Test coverage**
- A complete ITCase suite (`Flink23*ITCase`) is added under the
`fluss-flink-2.3` module, covering catalog, metrics, procedure, authorization,
sink, source (including binlog/changelog virtual tables, delta join, failover),
and tiering.
- `Flink23MultipleParameterToolTest` is added to validate
`MultipleParameterToolAdapter` behavior.
- `FlinkCatalogTest` and `Flink22CatalogTest` are updated for the
adapter-related cases.
### Tests
- Unit tests:
- `Flink23MultipleParameterToolTest`
- `Flink23CatalogTest`
- `Flink23TieringCommitOperatorTest`
- `FlinkCatalogTest`, `Flink22CatalogTest` (adapter-related cases updated)
- Integration tests (ITCases, all under the `fluss-flink-2.3` module):
- `Flink23CatalogITCase`, `Flink23MaterializedTableITCase`
- `Flink23MetricsITCase`
- `Flink23ProcedureITCase`
- `Flink23AuthorizationITCase`
- `Flink23ComplexTypeITCase`, `Flink23TableSinkITCase`,
`Flink23UndoRecoveryITCase`
- `Flink23BinlogVirtualTableITCase`, `Flink23ChangelogVirtualTableITCase`,
`Flink23DeltaJoinITCase`, `Flink23TableSourceBatchITCase`,
`Flink23TableSourceFailOverITCase`, `Flink23TableSourceITCase`
- `Flink23TieringITCase`
### API and Format
No breaking changes to the public API:
### Documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
