[
https://issues.apache.org/jira/browse/BEAM-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260696#comment-16260696
]
Etienne Chauchot commented on BEAM-3201:
----------------------------------------
[~chet.aldrich]: jackson JSONObject is not serializable, you can parse the json
string. you can use the jackson library that is already included in the io
dependencies. In theory, the IO is annotated with
{{@Experimental(Experimental.Kind.SOURCE_SINK)}} so we could change the
signature, but I'm not convinced it's worth the braking change. Another thing,
I needed a PMC to add you to the list of contributors so that I can assign you
the ticket. [~iemejia] just did that.
[~nerdynick]: _type and _index bulk metadata are not set in
{{WriteFn.processElement}} because they are set in the REST call. IMHO we
should not polluate the payload of documents with metadata fields. That said,
if you want to have dynamic index/type (meaning do not use
{{ConnectionConfiguration.withIndex}} and {{ConnectionConfiguration.withType}})
and also dynamic id depending of the document itself, we should add 3 optional
user defined functions so that the user can provide them. I guess it makes the
{{withDocumentIdField(String fieldName)}} redundant. So we should not implement
it.
> ElasticsearchIO should deal with documents id
> ---------------------------------------------
>
> Key: BEAM-3201
> URL: https://issues.apache.org/jira/browse/BEAM-3201
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Etienne Chauchot
> Assignee: Etienne Chauchot
>
> Today the ESIO only inserts the payload of the ES documents. Elasticsearch
> generates a document id for each record inserted. So each new insertion is
> considered as a new document. Users want to be able to update documents using
> the IO. So, for the write part of the IO, users should be able to provide a
> document id so that they could update already stored documents. Providing an
> id for the documents could also help the user on indempotency.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
