ASF GitHub Bot commented on FLINK-8101:

Github user jdewald commented on the issue:

    We were in the process of testing the Flink integration with the ES High 
Level Rest client in order to perform upgrades to ES6 and stumbled on some 
issues related to the bulk rejection retry handling. In short, swapping in the 
HLR backend will not perform retries correctly and it is not actually safe to 
simply call `add` on the `RequestIndexer` that is sent into the 
`ActionRequestFailureHander` from the Flink-side sink due to some 
synchronization issues. 
    I have created a bug report that has been accepted on the Elasticsearch 
side https://github.com/elastic/elasticsearch/issues/28885 to automatically 
perform retries.
    One additional note however is that in the case where the retries have been 
used up from the `BackoffPolicy`, using the `RetryRejectionFailureHandler` 
appears to cause a deadlock, so this may be something to fix on the 
ElasticsearchSink side. But it may be necessary to independently confirm that 
in case that is an issue particular to my testing. (TL;DR: The 
RetryRejectionFailureHandler is not safe to use with HLR and I am not certain 
it's safe to use with the `TransportClient` based one either in the case that 
it actually triggers). 

> Elasticsearch 6.x support
> -------------------------
>                 Key: FLINK-8101
>                 URL: https://issues.apache.org/jira/browse/FLINK-8101
>             Project: Flink
>          Issue Type: New Feature
>          Components: ElasticSearch Connector
>    Affects Versions: 1.4.0
>            Reporter: Hai Zhou UTC+8
>            Assignee: Flavio Pompermaier
>            Priority: Major
>             Fix For: 1.5.0
> Recently, elasticsearch 6.0.0 was released: 
> https://www.elastic.co/blog/elasticsearch-6-0-0-released  
> The minimum version of ES6 compatible Elasticsearch Java Client is 5.6.0

This message was sent by Atlassian JIRA

Reply via email to