[
https://issues.apache.org/jira/browse/FLINK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225557#comment-17225557
]
Leonard Xu commented on FLINK-19644:
------------------------------------
After discussed with [~lzljs3620320] [~lirui] [~ykt836]
We suggest to introduce only one option '*streaming-source.partition.include*'
and also combines with already introduced concepts like "partition-time" and
"create-time",the supported option are `all` and `latest`, the `all` means read
all partitions; the `latest` means read latest partition in order of
'streaming-source.partition.order', the `latest` only works` when the streaming
hive source table used as temporal table. By default the option is `all`.
Flink supports temporal join the latest hive partition by enabling
'streaming-source.monitor-interval' and setting
'streaming-source.partition.include' to 'latest', at the same time, user can
assign the partition compare order and data update interval by configuring
following partition-related options.
please see[1] for more details, I'll update the PR base on[1] soon
[1]https://docs.google.com/document/d/1ooQ1wvK-PR0MnCVG1MXGdt2FGIBoWdgdXlk6kO3RUSs/edit?usp=sharing
> Support read specific partition of Hive table in temporal join
> --------------------------------------------------------------
>
> Key: FLINK-19644
> URL: https://issues.apache.org/jira/browse/FLINK-19644
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Hive, Table SQL / Ecosystem
> Reporter: Leonard Xu
> Assignee: Leonard Xu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> It's a common case to use hive partitioned table as dimension table.
> Currently Hive table only supports load all data, It will be helpful if we
> can support read user specific partition in temporal table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
