[
https://issues.apache.org/jira/browse/FLINK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219071#comment-17219071
]
Jark Wu commented on FLINK-19644:
---------------------------------
First of all, I think they are two features: partition pruning and temporal
join *latest* partition.
IIRC, almost all the use cases now are loading the latest partition for
temporal join. We haven't seen any use cases that there is a secondary
partition key in addition to the time partition keys. Therefore, I would
suggest to only support the most common case, and leave the secondary partition
key feature to the future.
We can provide "read.partition=max_partition()" and
"read.reload-interval=1hour" for this case. Note, the options should be
compatible when we migrate the implementation to scan mode in the future.
Regarding to 'lookup.join.partition' = 'pt_year=2020;pt_month=09;pt_day=15',
IIUC, this option is used to prune partitions for lookup source? Why not
support this feature through WHERE condition and {{SupportsPartitionPushDown}}
interface?
> Support read specific partition of Hive table in temporal join
> --------------------------------------------------------------
>
> Key: FLINK-19644
> URL: https://issues.apache.org/jira/browse/FLINK-19644
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Hive, Table SQL / Ecosystem
> Reporter: Leonard Xu
> Assignee: Leonard Xu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> It's a common case to use hive partitioned table as dimension table.
> Currently Hive table only supports load all data, It will be helpful if we
> can support read user specific partition in temporal table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
