[
https://issues.apache.org/jira/browse/FLINK-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547751#comment-16547751
]
ASF GitHub Bot commented on FLINK-7386:
---------------------------------------
Github user cjolif commented on the issue:
https://github.com/apache/flink/pull/6043
@tzulitai
Happy to see progress is made on this!
> After merging this, I'll also try cherry-picking your 6.x REST-based ES
connector on top. If that works well, will also merge that.
Note that since the initial ES PR (#5374 ) I made a couple of changes in
our own copy of this.
1. Elasticsearch REST API can have a context root in addition the to list
of httpHosts, so I added the ability to have prefixPath, and calling:
```java
final RestClientBuilder builder =
RestClient.builder(httpHosts.toArray(new HttpHost[httpHosts.size()]));
if (pathPrefix != null && !pathPrefix.isEmpty() &&
!pathPrefix.equals("/")) {
builder.setPathPrefix(this.pathPrefix);
}
```
So that is set on the builder.
2. Elasticsearch REST can be protected by login/password so I added the
ability to set username/password:
```java
private CredentialsProvider getCredentialProvider() {
CredentialsProvider credentialsProvider = null;
if (userConfig.containsKey(CONFIG_KEY_ES_USERNAME) &&
userConfig.containsKey(CONFIG_KEY_ES_PASSWORD)) {
credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new
UsernamePasswordCredentials(userConfig.get(CONFIG_KEY_ES_USERNAME),
userConfig.get(CONFIG_KEY_ES_PASSWORD)));
}
return credentialsProvider;
}
```
and
then
```
builder.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder
.setDefaultCredentialsProvider(getCredentialProvider()))
```
More generally it should be easy for the user to change how the builder is
configure to make sure people can customize this as they want (like configure
SSL...).
> Flink Elasticsearch 5 connector is not compatible with Elasticsearch 5.2+
> client
> --------------------------------------------------------------------------------
>
> Key: FLINK-7386
> URL: https://issues.apache.org/jira/browse/FLINK-7386
> Project: Flink
> Issue Type: Improvement
> Components: ElasticSearch Connector
> Reporter: Dawid Wysakowicz
> Assignee: Fang Yong
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0
>
>
> In Elasticsearch 5.2.0 client the class {{BulkProcessor}} was refactored and
> has no longer the method {{add(ActionRequest)}}.
> For more info see: https://github.com/elastic/elasticsearch/pull/20109
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
