[
https://issues.apache.org/jira/browse/FLINK-28838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aitozi updated FLINK-28838:
---------------------------
Description:
When using the new source api, I found that if the source has no data, it still
brings high cpu usage.
The reason behind this is that it will always return the
{{RecordsWithSplitIds}} from the {{splitReader.fetch}} in FetchTask and it will
be added to the elementQueue. It will make the consumer be notified to wake up
frequently.
This causes the thread to keep busy to run and wake up, which leads to the high
sys and user cpu usage.
I think not all the SplitReader#fetch will block until there is data, if it
returns immediately when there is no data, then this problem will happen
was:
When using the new source api, I found that if the source has no data, it still
brings high cpu usage.
The reason behind this is that it will always return the
{{RecordsWithSplitIds}} from the {{splitReader.fetch}} in FetchTask and it will
be added to the elementQueue. It will make the consumer be notified to wake up
frequently.
the top thread shown by arthas
> Avoid to notify the elementQueue consumer when the fetch result is empty
> ------------------------------------------------------------------------
>
> Key: FLINK-28838
> URL: https://issues.apache.org/jira/browse/FLINK-28838
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common
> Affects Versions: 1.15.0, 1.15.1
> Reporter: Aitozi
> Priority: Major
> Attachments: 20220805165441.jpg
>
>
> When using the new source api, I found that if the source has no data, it
> still brings high cpu usage.
> The reason behind this is that it will always return the
> {{RecordsWithSplitIds}} from the {{splitReader.fetch}} in FetchTask and it
> will be added to the elementQueue. It will make the consumer be notified to
> wake up frequently.
> This causes the thread to keep busy to run and wake up, which leads to the
> high sys and user cpu usage.
> I think not all the SplitReader#fetch will block until there is data, if it
> returns immediately when there is no data, then this problem will happen
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
