Re: Upserts with Flink-elasticsearch

2016-03-31 Thread HungChang
Without indexRequest ES2 throws `document does not exit exception`. Based on https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-update.html#java-docs-update-api-upsert the upsert works althrough not sure it's the best way. return new UpdateRequest()

Re: Upserts with Flink-elasticsearch

2016-03-31 Thread Aljoscha Krettek
@suneel if it is possible to add upsert support for ES 1.x I would go for it, yes. On Tue, 29 Mar 2016 at 16:36 Zach Cox wrote: > You can just create a new UpdateRequest instance directly using its > constructor [1] like this: > > return new UpdateRequest() > .index(index

Re: Upserts with Flink-elasticsearch

2016-03-29 Thread Zach Cox
You can just create a new UpdateRequest instance directly using its constructor [1] like this: return new UpdateRequest() .index(index) .type(type) .id(element) .source(json); [1] http://javadoc.kyubu.de/elasticsearch/HEAD/or

Re: Upserts with Flink-elasticsearch

2016-03-29 Thread HungChang
Hi Zach, For using upsert in ES2, I guess it looks like as follows? However I cannot find which method in Request returns UpdateRequest while Requests.indexRequest() returns IndexRequest. Can I ask did you know it? public static UpdateRequest updateIndexRequest(String element) { Map json

Re: Upserts with Flink-elasticsearch

2016-03-28 Thread Suneel Marthi
Would it be useful to modify the existing Elasticsearch 1x sink to be able to handle Upserts ? On Mon, Mar 28, 2016 at 5:32 PM, Zach Cox wrote: > Hi Madhukar - with the current Elasticsearch sink in Flink 1.0.0 [1], I > don't think an upsert is possible, since IndexRequestBuilder can only > ret

Re: Upserts with Flink-elasticsearch

2016-03-28 Thread Zach Cox
Hi Madhukar - with the current Elasticsearch sink in Flink 1.0.0 [1], I don't think an upsert is possible, since IndexRequestBuilder can only return an IndexRequest. In Flink 1.1, the Elasticsearch 2.x sink [2] provides a RequestIndexer [3] that you can pass an UpdateRequest to do an upsert. Than