[
https://issues.apache.org/jira/browse/FLINK-32891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755686#comment-17755686
]
Andriy Redko commented on FLINK-32891:
--------------------------------------
Thanks [~thebranchnotmerged] , I think it could be useful enhancements,
[~martijnvisser] if you have no objections, please assign it to me
> Opensearch SQL connector crash job on upsert from multiple sources (409
> version conflict)
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-32891
> URL: https://issues.apache.org/jira/browse/FLINK-32891
> Project: Flink
> Issue Type: Bug
> Affects Versions: opensearch-1.0.1
> Reporter: Kobe Fitussi
> Priority: Critical
>
> Using Opensearch SQL Connector for flink , An attempt to perform an Upsert
> for the same document ID from multiple jobs at the same time has resulted in
> job crush with 409 version conflict error message, In our environment we
> cannot guarantee that messages will arrive separately.
> I suggest in RowOpensearchEmitter.processUpsert() , when an UpdateRequest
> object is being created it will also call an existing method of the
> UpdateRequest "retryOnConflict(int retries)" which is designed to remedy this
> issue where 'retries' will be set by OpensearchConnectorOptions class ,
> default 0.
> suggested parameter name : 'sink.upsert.retry-on-conflict'
> Also, I do not see evidence of bulk-retries being Effective in this case
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
