[
https://issues.apache.org/jira/browse/HUDI-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sagar Sumit updated HUDI-3816:
------------------------------
Description:
Please see this comment on the PR:
[https://github.com/trinodb/trino/pull/10228#issuecomment-1087291399]
Currently, while getting next bactch of splits, the split loader is idle until
there is something in split queue (class HudiSplitSource). As suggested in the
PR comment, we can avoid this blocking implementation and instead use an async
queue and return uncompleted future.
was:
Please see this comment on the PR:
[https://github.com/trinodb/trino/pull/10228#issuecomment-1087291399]
Currently, while getting next bactch of splits, the split loader is idle until
there is something in split queue. As suggested in the PR comment, we can avoid
this blocking implementation and instead use an async queue and return
uncompleted future.
> Change HudiSplitSource to use async queue and shared executor service
> ---------------------------------------------------------------------
>
> Key: HUDI-3816
> URL: https://issues.apache.org/jira/browse/HUDI-3816
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Sagar Sumit
> Priority: Major
>
> Please see this comment on the PR:
> [https://github.com/trinodb/trino/pull/10228#issuecomment-1087291399]
> Currently, while getting next bactch of splits, the split loader is idle
> until there is something in split queue (class HudiSplitSource). As suggested
> in the PR comment, we can avoid this blocking implementation and instead use
> an async queue and return uncompleted future.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
