[
https://issues.apache.org/jira/browse/FLINK-32887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-32887:
-----------------------------------
Labels: pull-request-available (was: )
> SourceCoordinatorContext#workerExecutor may cause task initializing slowly
> ---------------------------------------------------------------------------
>
> Key: FLINK-32887
> URL: https://issues.apache.org/jira/browse/FLINK-32887
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common, Runtime / Coordination
> Affects Versions: 1.15.2
> Reporter: liang jie
> Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> SourceCoordinatorContext#workerExecutor is typically used to calculate
> partitions of a source task and is implemented by a ScheduledExecutorService
> with only 1 core (hard coded).Tasks to calculate partitions with be executed
> through the function 'workerExecutor.scheduleAtFixedRate'.
> ---
> In some case, for example, 'getSubscribedPartitions' method will take quite a
> long time(e.g. 5min) because of lots of topics are included in the same task
> or requests to outer systems timeout etc. And partitionDiscoveryInterval is
> set to a short intervel e.g. 1min.
> In this case, 'getSubscribedPartitions' runnable tasks will be triggered
> repeatedly and be queued in the queue of workerExecutor, during the first
> 'getSubscribedPartitions' task running duration, which causing
> 'checkPartitionChanges' tasks will be queued too. Each
> 'checkPartitionChanges' task needs to wait for 25mins(5 *
> 'getSubscribedPartitions' task execution duration) before it was executed.
> ---
> In my view, tasks of workerExecutor should be scheduled with fix deley rather
> than at fixed rate. Because there is no meaning that
> 'getSubscribedPartitions' tasks being repeatedly executed without a
> 'checkPartitionChanges' execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
