[
https://issues.apache.org/jira/browse/FLINK-33857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo closed FLINK-33857.
------------------------------
Fix Version/s: elasticsearch-3.1.0
Resolution: Fixed
main via(b43500510085ff37239efbb107179333df83ae6f)
> Expose BulkProcessor.flush via RequestIndexer to allow custom flush logic
> -------------------------------------------------------------------------
>
> Key: FLINK-33857
> URL: https://issues.apache.org/jira/browse/FLINK-33857
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
> Affects Versions: elasticsearch-3.0.1
> Reporter: Peter Schulz
> Assignee: Peter Schulz
> Priority: Major
> Labels: pull-request-available
> Fix For: elasticsearch-3.1.0
>
>
> There's [a bug|https://github.com/elastic/elasticsearch/issues/103406] in
> elasticsearch that may lead to underestimated bulk request sizes. We are hit
> by this bug and therefore, I would like to propose a simple improvement to
> work around it:
> {{ElasticsearchWriter}} uses {{RequestIndexer}} as a facade for ES'
> {{BulkProcessor}}. As of now, only the {{add(…)}} methods are delegated.
> Under the hood, the {{BulkProcessor}} takes care of flushing bulk requests to
> the server when any of the configurable limits is hit (there's another bug:
> flushing will happen _after_ the limit has exceeded).
> (+) *Proposal:* Expose {{BulkProcessor.flush()}} via
> {{RequestIndexer.flush()}}. This way we can easily implement logic to decide
> when to flush, inside the emitter.
> I created a [pull
> request|https://github.com/apache/flink-connector-elasticsearch/pull/85].
> Feedback is highly welcome. :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
