Kobe Fitussi created FLINK-32891:
------------------------------------
Summary: Opensearch SQL connector crash job on upsert from
multiple sources (409 version conflict)
Key: FLINK-32891
URL: https://issues.apache.org/jira/browse/FLINK-32891
Project: Flink
Issue Type: Bug
Affects Versions: opensearch-1.0.1
Reporter: Kobe Fitussi
Using Opensearch SQL Connector for flink , An attempt to perform an Upsert for
the same document ID from multiple jobs at the same time has resulted in job
crush with 409 version conflict error message, In our environment we cannot
guarantee that messages will arrive separately.
I suggest in RowOpensearchEmitter.processUpsert() , when an UpdateRequest
object is being created it will also call an existing method of the
UpdateRequest "retryOnConflict(int retries)" which is designed to remedy this
issue where 'retries' will be set by OpensearchConnectorOptions class , default
0.
suggested parameter name : 'sink.upsert.retry-on-conflict'
Also, I do not see evidence of bulk-retries being Effective in this case
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
