[
https://issues.apache.org/jira/browse/BEAM-12722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394089#comment-17394089
]
Evan Galpin commented on BEAM-12722:
------------------------------------
[~bdaghir] this should be available as of SDK 2.31.0 via
`ElasticsearchIO.write().withDocVersionFn(...)` and `.withDocVersionType()`[1].
Note that you may want to also make use of `.withIgnoreVersionConflicts()`[2]
so that if elements are delivered to Elasticsearch out-of-order from their
version numbers, docs with older versions will be attempted, fail, but not
block your pipeline with endless retries (due to "failure" in ES response).
[1]
[https://github.com/apache/beam/blob/master/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java#L1376-L1388]
[2]
https://github.com/apache/beam/blob/7bfeca2a6ceb4ef8ac3c9822a59d770490c1e6f9/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java#L1923-L1941
> Add ElasticsearchIO External Versioning
> ---------------------------------------
>
> Key: BEAM-12722
> URL: https://issues.apache.org/jira/browse/BEAM-12722
> Project: Beam
> Issue Type: Improvement
> Components: io-java-elasticsearch
> Environment: Java SDK (all environments)
> Reporter: Benjamin Daghir
> Priority: P2
> Labels: features
>
> h2. Elasticsearch External Versioning
> Beam's ElasticsearchIO Java SDK currently does not support [external
> versioning|https://www.elastic.co/blog/elasticsearch-versioning-support].
> h3. Use Case
> A primary use case is leveraging partial updates when performing a reindex
> operation. Without explicitly setting the external version, the versioning is
> set to default and version conflicts prevent any document updates.
> h3. API Usage for External Versioning
> When using Elasticsearch's API, the *?version_type=external&version=1.0.0*
> query parameters must be provided.
> [The options for *version_type*
> are|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-version-types]:
> * *external* or *external_gt*
> * *external_gte*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
