[
https://issues.apache.org/jira/browse/FLINK-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323138#comment-17323138
]
Flink Jira Bot commented on FLINK-15206:
----------------------------------------
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> support dynamic catalog table for truly unified SQL job
> -------------------------------------------------------
>
> Key: FLINK-15206
> URL: https://issues.apache.org/jira/browse/FLINK-15206
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
> Reporter: Bowen Li
> Assignee: Bowen Li
> Priority: Major
> Labels: stale-assigned
>
> currently if users have both an online and an offline job with same business
> logic in Flink SQL, their codebase is still not unified. They would keep two
> SQL statements whose only difference is the source (or/and sink) table (with
> different params). E.g.
> {code:java}
> // online job
> insert into x select * from kafka_table (starting time) ...;
> // offline backfill job
> insert into x select * from hive_table (starting and ending time) ...;
> {code}
> We can introduce a "dynamic catalog table". The dynamic catalog table acts as
> a view, and is just an abstract table of multiple actual tables behind it
> that can be switched under some configuration flags. When execute a job,
> depending on the configuration, the dynamic catalog table can point to an
> actual source table.
> A use case for this is the example given above - when executed in streaming
> mode, {{my_source_dynamic_table}} should point to a kafka catalog table with
> a new starting position, and in batch mode, {{my_source_dynamic_table}}
> should point to a hive catalog table with starting/ending positions.
>
> One thing to note is that the starting position of kafka_table, and
> starting/ending position of hive_table are different every time. needs more
> thinking of how can we accommodate that
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
