[
https://issues.apache.org/jira/browse/BEAM-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493547#comment-16493547
]
Etienne Chauchot commented on BEAM-3199:
----------------------------------------
Nice ! Thanks for you work. When it is ready submit a PR and I'll do the
review.
Some comments:
* I'd like to keep one production module if possible to avoid confusion for
the users (keep the very same code and keep the very same pom). This was done
with ES2 and ES5 because only the split part of the IO was different between
the 2 versions (ES5 could leverage the slice API to comply to the runner
desiredBundleSize)
* Indeed better to use a high level ES client; it was not available at the
time I did IO v2 and v5
* ESIntegTestCase was not ideal at all, I had headaches using it for ES5
tests. And I ended up with a hack of the jarHell detection
* for IT we use docker or kubernates but for UT we need an embedded (in JVM)
because a mock is not enough to test some complicated parts (split, cache,
batching ...). In ES2 I could get a cluster from the client and for ES5 this
feature was removed so I had to use ESIntegTestCase. I don't know the embedded
server tests capabilities of ES6.
* About the use of json in the elements of the PCollection: we had
discussions about using POJO but we decided not to (see
[https://github.com/apache/beam/pull/4956] my comment of Apr 6th). I'd like to
here your thoughts about this.
> Upgrade to Elasticsearch 6.x
> ----------------------------
>
> Key: BEAM-3199
> URL: https://issues.apache.org/jira/browse/BEAM-3199
> Project: Beam
> Issue Type: Improvement
> Components: io-java-elasticsearch
> Reporter: Jean-Baptiste Onofré
> Assignee: Jeroen Steggink
> Priority: Major
>
> Elasticsearch 6.x is now GA. As it's fully compatible with Elasticsearch 5.x,
> it makes sense to upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
