[
https://issues.apache.org/jira/browse/FLINK-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiangjie Qin updated FLINK-20108:
---------------------------------
Description:
Currently the split fetchers are only removed from the
{{SplitFetcherManager.fetchers}} when the thread exit. The may cause problem
because when {{SplitFetcherManager.addSplits()}} is called, it may see a
shutting down split fetcher and adds splits to it. These splits will then just
be lost.
This issue is actually already fixed in FLINK-18128. The fix needs to
cherry-picked to 1.11.3
was:
Currently the split fetchers are only removed from the
{{SplitFetcherManager.fetchers}} when the thread exit. The may cause problem
because when {{SplitFetcherManager.addSplits()}} is called, it may see a
shutting down split fetcher and adds splits to it. These splits will then just
be lost.
The fix is to make {{SplitFetcherManager.fetchers}} field private and introduce
a protected {{addSplitsToFetcher(Splits, fetcherId)}} method. That method will
ensure the splits won't be assigned to a shutting down split fetcher instead of
a new fetcher.
> SingleThreadFetcherManager may add splits to a shutting down SplitFetcher
> -------------------------------------------------------------------------
>
> Key: FLINK-20108
> URL: https://issues.apache.org/jira/browse/FLINK-20108
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Affects Versions: 1.11.2
> Reporter: Jiangjie Qin
> Assignee: Jiangjie Qin
> Priority: Major
> Fix For: 1.11.3
>
>
> Currently the split fetchers are only removed from the
> {{SplitFetcherManager.fetchers}} when the thread exit. The may cause problem
> because when {{SplitFetcherManager.addSplits()}} is called, it may see a
> shutting down split fetcher and adds splits to it. These splits will then
> just be lost.
> This issue is actually already fixed in FLINK-18128. The fix needs to
> cherry-picked to 1.11.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
