[
https://issues.apache.org/jira/browse/FLINK-19798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220537#comment-17220537
]
Nicholas Jiang commented on FLINK-19798:
----------------------------------------
[~FrankZou], Hive source would be refactored through new Source interface based
on FLIP-27. This improvement could be added on new Hive source in Flink 1.12
version.
> Support creating hive table input splits in parallel
> ----------------------------------------------------
>
> Key: FLINK-19798
> URL: https://issues.apache.org/jira/browse/FLINK-19798
> Project: Flink
> Issue Type: Improvement
> Reporter: Zou
> Priority: Major
>
> For now, we could only creating hive table input splits serially.
> HiveTableInputFormat::createInputSplits cost 40s~60s for a table with 1500
> partition in my tests. IMO, we could support creating splits in parallel,
> which will save most of the time to do this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
