[
https://issues.apache.org/jira/browse/FLINK-39732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongshun Wang reassigned FLINK-39732:
-------------------------------------
Assignee: Hongshun Wang
> Introduce TableDiscoverer SPI for flexible table subscription (with default
> JdbcTableDiscoverer)
> ------------------------------------------------------------------------------------------------
>
> Key: FLINK-39732
> URL: https://issues.apache.org/jira/browse/FLINK-39732
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.5.0
> Reporter: Hongshun Wang
> Assignee: Hongshun Wang
> Priority: Major
> Fix For: cdc-3.6.0
>
>
> h3. Background
> Currently, Flink CDC only supports subscribing to source tables via a pattern
> (regex/glob) syntax. While this works well in many cases, it is not flexible
> enough for users who want to dynamically discover the tables to capture from
> the source database itself — for example, by querying the catalog of a
> PostgreSQL or MySQL instance, filtering by tags/owners, or applying
> business-specific selection rules.
>
> h3. Motivation
> * Users often want to subscribe to "all tables in database X that match some
> runtime condition", which cannot be expressed cleanly with a static pattern.
> * Different databases (MySQL, PostgreSQL, Oracle, SQL Server, MongoDB, etc.)
> and different organizations have different rules for what "the right set of
> tables" means.
> * A pluggable discovery layer would let users (and connector authors) extend
> subscription behavior without touching CDC core code.
>
> h3. Proposal
> Introduce a new SPI: TableDiscoverer, responsible for resolving the concrete
> list of tables the CDC job should capture at job startup (and, optionally,
> periodically for newly created tables).
>
> public interface TableDiscoverer extends Serializable \{
> /** Identifier used in connector options, e.g. "pattern", "jdbc",
> "custom-x". */
> String identifier(); /** Discover tables based on user configuration
> and source connection info. */
> List<TableId> discover(TableDiscoveryContext context);
> }
>
>
> h3. Default implementation: JdbcTableDiscoverer
>
> A built-in JdbcTableDiscoverer gives MySQL, PostgreSQL, Oracle, SQL Server,
> etc. a sensible default discovery strategy out of the box, while still
> letting each dialect override edge cases (e.g., PostgreSQL publication
> filtering, MySQL information_schema shortcuts).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
