[
https://issues.apache.org/jira/browse/FLINK-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657408#comment-15657408
]
ASF GitHub Bot commented on FLINK-4491:
---------------------------------------
GitHub user ddolzan opened a pull request:
https://github.com/apache/flink/pull/2790
[FLINK-4491] Handle index.number_of_shards in the ES connector
Implemented the Index Template and Index Mapping creation.
Number of shards and many other properties can be defined in the Index
Template.
### Usage
Before calling ElasticasearchSink instantiate ElasticSearchHelper
```java
ElasticSearchHelper esHelper = new ElasticSearchHelper(config, transports);
//Create an Index Template given a name and the json structure
esHelper.initTemplate(templateName, templateRequest);
//Create an Index Mapping given the Index Name, DocType and the json
structure
esHelper.initIndexMapping(indexName, docType, mappingsRequest);
```
### TemplateRequest example
```json
{
"template": "te*",
"settings": {
"number_of_shards": 1
},
"mappings": {
"type1": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z YYYY"
}
}
}
}
}
```
### MappingRequest example
```json
{
"mappings": {
"user": {
"_all": {
"enabled": false
},
"properties": {
"title": {
"type": "string"
},
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
}
}
}
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ddolzan/flink issue-4491
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2790.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2790
----
commit bae4237b755c1522a026ec691388ec8c197e8ab8
Author: ddolzan <[email protected]>
Date: 2016-11-11T15:21:05Z
[FLINK-4491] Added index template and mappings creation
----
> Handle index.number_of_shards in the ES connector
> -------------------------------------------------
>
> Key: FLINK-4491
> URL: https://issues.apache.org/jira/browse/FLINK-4491
> Project: Flink
> Issue Type: Improvement
> Components: Streaming Connectors
> Affects Versions: 1.1.0
> Reporter: Flavio Pompermaier
> Priority: Minor
> Labels: elasticsearch, streaming
>
> At the moment is not possible to configure the number of shards if an index
> does not already exists on the Elasticsearch cluster. It could be a great
> improvement to handle the index.number_of_shards (passed in the configuration
> object). E.g.:
> {code:java}
> Map<String, String> config = Maps.newHashMap();
> config.put("bulk.flush.max.actions", "1");
> config.put("cluster.name", "my-cluster-name");
> config.put("index.number_of_shards", "1");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
