[
https://issues.apache.org/jira/browse/FLINK-29805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caizhi Weng closed FLINK-29805.
-------------------------------
Resolution: Fixed
master: 0a193a8110aa4716250fed3c4223018ab519c9b1
release-0.2: d41d14a5921aa27ebd903eed45365841093e97ed
> Table Store sink continuously fails with "Trying to add file which is already
> added" when snapshot committing is slow
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-29805
> URL: https://issues.apache.org/jira/browse/FLINK-29805
> Project: Flink
> Issue Type: Bug
> Components: Table Store
> Affects Versions: table-store-0.3.0, table-store-0.2.2
> Reporter: Caizhi Weng
> Assignee: Caizhi Weng
> Priority: Major
> Labels: pull-request-available
> Fix For: table-store-0.3.0, table-store-0.2.2
>
>
> Table Store sink continuously fails with "Trying to add file which is already
> added" when snapshot committing is slow.
> This is due to a bug in {{FileStoreCommitImpl#filterCommitted}}. When this
> method finds an identifier, it removes the identifier from a map. However
> different snapshots may have the same identifier (for example an APPEND
> commit and the following COMPACT commit will have the same identifier), so we
> need to use another set to check for identifiers.
> When snapshot committing is fast there is at most 1 identifier to check after
> the job restarts, so nothing happens. However when snapshot committing is
> slow, there will be multiple identifiers to check and some identifiers will
> be mistakenly kept.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
